Files

T

yuxuanhui 7797ff88df Add comprehensive documentation and templates for LangChain skill development

- Introduced a new reference document for streaming output issues, detailing the differences between streaming APIs and providing solutions for common problems.
- Created a structured output issues reference, outlining the use of `with_structured_output`, `response_format`, and schema enforcement strategies.
- Added a user query convention guide to standardize structured questions for skill authors, including block types for queries and data gathering.
- Implemented a template for a chat model, encapsulating API key management, request payload construction, and response handling.
- Established a symlink for the LangChain dev guide in the Claude skills directory for easier access.
- Initialized a skills lock file to manage dependencies and versions for the LangChain dev guide.

2026-09-07 16:19:11 +08:00

3.0 KiB

Raw Permalink Blame History

Chat Model Integration Tests

After writing chat_model.py, you must write integration tests to verify the model class works correctly.

Test Framework

Use ChatModelIntegrationTests from langchain_tests as the base class, and run tests with pytest.

Install dependencies:

pip install langchain-tests pytest python-dotenv

Test File Structure

Place test files following standard unit test directory conventions:

src/<models_dir>/<model_name>/
├── __init__.py
├── chat_model.py
└── ...

tests/
└── test_chat_<model_name>.py

Standard Test Class

For a new provider (e.g., Qwen, GLM), create a test class that inherits from ChatModelIntegrationTests and provides the following properties:

from __future__ import annotations

import pytest
from dotenv import load_dotenv
from langchain_core.language_models import BaseChatModel
from langchain_tests.integration_tests import ChatModelIntegrationTests

from models.qwen.chat_model import ChatQwen  # replace with the actual import path

load_dotenv()


class TestChatQwen(ChatModelIntegrationTests):
    @property
    def chat_model_class(self) -> type[BaseChatModel]:
        return ChatQwen

    @property
    def chat_model_params(self) -> dict:
        return {
            "model": "qwen-plus",
            "temperature": 0,
        }

Required Properties

Property	Description
`chat_model_class`	Returns the chat model class under test.
`chat_model_params`	Parameters for creating an instance. Must include `model`; `temperature: 0` is recommended for deterministic results.

Running Tests

# Run tests for a single model
pytest tests/test_chat_<model_name>.py -v

# Skip tests marked as xfail (run only expected passes)
pytest tests/test_chat_<model_name>.py -v -m "not xfail"

# Run all model tests
pytest tests/ -v

Common Issues

After setting up the test, you will likely encounter the following issues. Address them before concluding tests pass.

Model package not importable

By default, the <models_dir>/ directory is not installed as a Python package, so from <models_dir>.xxx import ... in tests will fail. Two changes are needed in pyproject.toml:

1) Add build-system config:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["<models_dir>"]

2) Install in editable mode:

uv pip install -e .

Without this, pytest fails with ModuleNotFoundError: No module named '<models_dir>'.

Async tests not running (pytest-asyncio strict mode)

pytest-asyncio defaults to Mode.STRICT, which requires every async test to have an @pytest.mark.asyncio decorator. langchain_tests async methods lack this decorator.

Add to pyproject.toml:

[tool.pytest.ini_options]
asyncio_mode = "auto"

Without this, all async tests (test_ainvoke, test_astream, test_abatch, etc.) fail with "async def functions are not natively supported."

3.0 KiB Raw Permalink Blame History