``llm`` module
================================

The ``llm`` module of the ``ml_toolkit`` contains all the functions that enable LLM usage within Databricks notebooks.
We expose three functions:

* ``run_llm_batch``: performs row-level LLM querying in our data, outputting their response to a new column.
* ``estimate_token_usage``: estimates token usage of the ``run_llm_batch`` function.
* ``query``: gives you the ability to directly query a model and give it context and tools to perform an action.

.. attention:: These functions are in an experimental phase and are subject to change. If you have any feedback,
                please submit it via our `Jira Form <https://get.support.yipitdata.com/servicedesk/customer/portal/139/group/260/create/609>`_.

.. attention:: You **must** pass a ``cost_component_name`` to the functions that call the LLMs, otherwise they will
                raise exceptions.

Quota Controls
^^^^^^^^^^^^^^^

We have set strict usage controls, or quotas, to limit usage. This means you will only be able to run a set limit of
tokens without having an exception being raised. LLMs can get very expensive quickly, so these limits are set to avoid
high usage.

If you have a valid usecase that solves a business problem and needs a higher quota to run that with your data, please
submit a ticket. The approval process for this involves getting your manager to approve your usecase and requested budget.
When submitting this, always include the output from ``estimate_token_usage``.


Available Models
^^^^^^^^^^^^^^^^^

The following models are available:

    * ``databricks-meta-llama-3-1-8b-instruct``
    * ``databricks-meta-llama-3-3-70b-instruct``
    * ``databricks-meta-llama-3-1-405b-instruct``
    * ``databricks-llama-4-maverick``
    * ``databricks-claude-3-7-sonnet``
    * ``gpt-4o``
    * ``gpt-4o-mini``
    * ``gpt-4o-search-preview`` (only in ``query``)
    * ``gpt-4o-mini-search-preview`` (only in ``query``)


.. attention:: We strongly suggest using the ``llama`` models, as they are considerably cheaper than the openAI models.


Functions
^^^^^^^^^^

run_llm_batch
^^^^^^^^^^^^^

.. autoapifunction:: ml_toolkit.functions.llm.run_llm_batch

estimate_token_usage
^^^^^^^^^^^^^^^^^^^^

.. autoapifunction:: ml_toolkit.functions.llm.estimate_token_usage

query
^^^^^

.. autoapifunction:: ml_toolkit.functions.llm.query

LLMResponse
^^^^^^^^^^^

.. autoclass:: ml_toolkit.functions.llm.query.LLMResponse