pangukitsappdev.cache package

Submodules

pangukitsappdev.cache.gptcache_adapter module

class pangukitsappdev.cache.gptcache_adapter.CSSVectorStore(vector_config: VectorStoreConfig)

基类：VectorBase

适配GPTCache的VectorBase的类，用来集成进gptcache作为向量存储使用

Attributes:

_top_k: 默认的4，从CSS中检索出的文档数量 css_vector_search: 内部分装的CSS向量存储 vector_config: 用于构造实例的配置内容

delete(ids) → bool

flush()

mul_add(datas: List[VectorData])

rebuild(ids=None) → bool

search(data: ndarray, top_k: int)

相似检索 Args:

data: 向量数据 top_k: 检索top_k个数据

Returns:

（score: float, id: int）

class pangukitsappdev.cache.gptcache_adapter.SemanticGptCacheApi(cache_config: CacheStoreConfig)

基类：CacheApiAdapter

GptCache语义缓存 基于GptCache组件做的适配和扩展。集成了CSS作为vector_store

pangukitsappdev.cache.gptcache_adapter.add_text_request(index_name: str, vector: List[float], content: str, doc_metadata: dict) → Tuple[str, dict]

这里直接使用content作为es的id

pangukitsappdev.cache.gptcache_adapter.get_doc_with_score_func(hit: dict) → Tuple[Document, float]

pangukitsappdev.cache.gptcache_adapter.text_mapping(dim: int, metric: str = 'inner_product') → Dict

pangukitsappdev.cache.in_memory module

class pangukitsappdev.cache.in_memory.InMemoryCacheApi(cache_config: CacheStoreConfig)

基类：CacheApiAdapter

class pangukitsappdev.cache.in_memory.TTLInMemoryCache(ttl, maximum_size)

基类：BaseCache

clear(**kwargs: Any) → None

Clear cache that can take additional keyword arguments.

lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]]

Look up based on prompt and llm_string.

A cache implementation is expected to generate a key from the 2-tuple of prompt and llm_string (e.g., by concatenating them with a delimiter).

Args:

prompt: a string representation of the prompt.

In the case of a Chat model, the prompt is a non-trivial serialization of the prompt into the language model.

llm_string: A string representation of the LLM configuration.

This is used to capture the invocation parameters of the LLM (e.g., model name, temperature, stop tokens, max tokens, etc.). These invocation parameters are serialized into a string representation.

Returns:

On a cache miss, return None. On a cache hit, return the cached value. The cached value is a list of Generations (or subclasses).

update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None

Update cache based on prompt and llm_string.

The prompt and llm_string are used to generate a key for the cache. The key should match that of the look up method.

Args:

prompt: a string representation of the prompt.

In the case of a Chat model, the prompt is a non-trivial serialization of the prompt into the language model.

llm_string: A string representation of the LLM configuration.

This is used to capture the invocation parameters of the LLM (e.g., model name, temperature, stop tokens, max tokens, etc.). These invocation parameters are serialized into a string representation.

return_val: The value to be cached. The value is a list of Generations

(or subclasses).

pangukitsappdev.cache.sql module

class pangukitsappdev.cache.sql.MySQLLLMCacheSchema(**kwargs)

基类：Base

MySQL table for full LLM Cache (all generations).

answer

create_date

id

prompt

prompt_prefix

session_tag

class pangukitsappdev.cache.sql.SQLCache(engine: Engine, cache_schema, ttl, maximum_size, pool_size=5)

基类：SQLAlchemyCache

clear(**kwargs: Any) → None

Clear cache.

delete_by_batch(llm_string: str)

evict(prompt, session_tag)

lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]]

Look up based on prompt and llm_string.

update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None

Update based on prompt and llm_string.

class pangukitsappdev.cache.sql.SqlCacheApi(cache_config: CacheStoreConfig)

基类：CacheApiAdapter

pangukitsappdev.cache.ttl_redis module

class pangukitsappdev.cache.ttl_redis.RedisCacheApi(cache_config: CacheStoreConfig)

基类：CacheApiAdapter

支持TTL的缓存策略，内部封装Langchain的redis缓存 1. 通过CacheStoreConfig创建Redis实例，从而初始化RedisCache

class pangukitsappdev.cache.ttl_redis.RedisSemanticCacheApi(cache_config: CacheStoreConfig)

基类：CacheApiAdapter

Redis的语义缓存 内部封装pangukitsappdev.cache.redis_ext.TTLRedisSemanticCache。支持配置缓存的ttl

class pangukitsappdev.cache.ttl_redis.TTLRedisCache(redis_client, ttl)

基类：RedisCache

支持TTL的缓存策略，继承自langchain的BaseCache

clear(**kwargs: Any)

Clear cache. If asynchronous is True, flush asynchronously.

update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None

Update cache based on prompt and llm_string.

class pangukitsappdev.cache.ttl_redis.TTLRedisSemanticCache(redis_url: str, embedding: Embeddings, ttl, score_threshold: float = 0.2)

基类：RedisSemanticCache

update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None

更新缓存，并配置缓存的ttl 逻辑复用父类逻辑，获取ids进行ttl配置 Args:

prompt: prompt query llm_string: 唯一标识llm return_val: llm返回的数据

Module contents