🤖

Thank you for your feature request. I understand that you would like to add metadata to a queryEngine or invocation in LlamaIndex, similar to the capability in LangChain. This metadata would then be made available to the callback system for better observability in production, troubleshooting, and automatic labeling.

Currently, LlamaIndex does support adding metadata to the queryEngine and invocations. This is evident from the metadata_mode parameter in the CitationQueryEngine class's __init__ method and from_args class method. The metadata_mode parameter is an instance of the MetadataMode class, which controls how metadata is included in the citation prompt. You can find the relevant code in the CitationQueryEngine class.

However, to make this metadata available to the callback system, you would need to create new event types for them in the CBEventType enumeration. Then, you can use the on_event_start and on_event_end methods to handle these events, passing the metadata as the payload argument. Here is an example of how you can add metadata to a queryEngine event:

from llama_index.callbacks.schema import CBEventType

# Define your metadata
query_engine_metadata = {
    'engine_id': 'engine_1',
    'engine_type': 'type_1',
    # Add more metadata as needed
}

# Start the queryEngine event
event_id = callback_manager.on_event_start(
    CBEventType.QUERY_ENGINE,  # Assuming QUERY_ENGINE is a new event type you added
    payload=query_engine_metadata
)

# ... Perform the queryEngine operations ...

# End the queryEngine event
callback_manager.on_event_end(
    CBEventType.QUERY_ENGINE,
    payload=query_engine_metadata,
    event_id=event_id
)

This will make the metadata available to the callback system, which can be used for better observability in production, troubleshooting, and automatic labeling. You can find more about this in the CallbackManager class.

Please note that this is a high-level suggestion and the actual implementation may vary based on your specific use case and the current structure of your code. If you need further assistance, feel free to ask.

Sources

• llama_index/callbacks/base.py
• llama_index/query_engine/citation_query_engine.py