diff --git a/README.md b/README.md index 651a627..79c9660 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,11 @@ AllEndpoints is a powerful Python module for making inferences with various LLM providers through a unified interface. It supports multiple providers including Ollama (local), HuggingFace, Together, Google Gemini, AIQL, Groq, NVIDIA, and GitHub Copilot APIs. +> **Quick Start**: With [uv](https://github.com/astral-sh/uv) installed, you can run AllEndpoints without explicit dependency installation: +> ```bash +> uv run allendpoints.py --list +> ``` + ## Table of Contents - [Installation](#installation) @@ -26,10 +31,24 @@ AllEndpoints is a powerful Python module for making inferences with various LLM cd allendpoints ``` -2. Install the required dependencies using the requirements.txt file: +2. Choose one of the following installation methods: + + ### Option A: Using pip + Install the required dependencies using the requirements.txt file: ```bash pip install -r requirements.txt ``` + Then run the script directly: + ```bash + python allendpoints.py [arguments] + ``` + + ### Option B: Using uv (Recommended) + If you have [uv](https://github.com/astral-sh/uv) installed, you can run the script without explicitly installing dependencies: + ```bash + uv run allendpoints.py [arguments] + ``` + This will automatically create a virtual environment and install all required dependencies on first run. 3. Install Ollama (optional, for local inference): - [Ollama Installation Guide](https://github.com/ollama/ollama) @@ -140,32 +159,56 @@ options: **List all available providers and models:** ```bash +# Using python directly python allendpoints.py --list + +# Using uv run +uv run allendpoints.py --list ``` **Run inference with a specific provider and model:** ```bash +# Using python directly python allendpoints.py "What is the capital of France?" --provider ollama --model llama3.2:3b + +# Using uv run +uv run allendpoints.py "What is the capital of France?" --provider ollama --model llama3.2:3b ``` **Run inference with a specific provider and its default model:** ```bash +# Using python directly python allendpoints.py "Explain quantum computing" --provider gemini + +# Using uv run +uv run allendpoints.py "Explain quantum computing" --provider gemini ``` **Run inference with a custom system prompt:** ```bash +# Using python directly python allendpoints.py "Write a poem about AI" --provider ollama --model llama3.2:3b --system "You are a poetic assistant." + +# Using uv run +uv run allendpoints.py "Write a poem about AI" --provider ollama --model llama3.2:3b --system "You are a poetic assistant." ``` **Run inference on all available providers and models:** ```bash +# Using python directly python allendpoints.py "What is the meaning of life?" -a + +# Using uv run +uv run allendpoints.py "What is the meaning of life?" -a ``` **Run with debug output:** ```bash +# Using python directly python allendpoints.py "How does a nuclear reactor work?" --provider nvidia --model qwen2.5-coder-32b --debug + +# Using uv run +uv run allendpoints.py "How does a nuclear reactor work?" --provider nvidia --model qwen2.5-coder-32b --debug ``` ## Using as a Python Module diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..fa63535 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,17 @@ +[project] +name = "allendpoints" +version = "0.1.0" +description = "A powerful Python module for making inferences with various LLM providers through a unified interface." +readme = "README.md" +requires-python = ">=3.12" +dependencies = [ + "ollama>=0.1.6", + "requests>=2.31.0", + "google-generativeai>=0.3.0", + "huggingface_hub>=0.19.0", + "together>=0.2.8", + "groq>=0.4.0", + "openai>=1.6.0", + "colorama>=0.4.6", + "python-dotenv>=1.0.0" +] \ No newline at end of file