> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/cvat-ai/cvat/llms.txt
> Use this file to discover all available pages before exploring further.

# Testing Guide

> Learn how to write and run tests in CVAT

Testing is crucial for maintaining CVAT's quality and reliability. This guide covers how to run existing tests and write new ones.

## Test Structure

CVAT has several test suites:

* **Python REST API tests**: `tests/python/rest_api/`
* **Python SDK tests**: `tests/python/sdk/`
* **Python CLI tests**: `tests/python/cli/`
* **Frontend unit tests**: In component directories
* **E2E tests**: `tests/cypress/e2e/`

## Prerequisites

Before running tests:

1. Ensure Docker is running
2. Stop any existing CVAT instances that might use the same ports
3. Install Python dependencies:

```bash theme={null}
pip install -r tests/python/requirements.txt
```

4. Install Node.js dependencies:

```bash theme={null}
cd tests
yarn install
```

## Running Python Tests

### REST API Tests

The REST API tests use pytest and run against a real CVAT instance in Docker containers.

#### Run All Tests

```bash theme={null}
pytest tests/python/
```

This command automatically:

* Starts all necessary Docker containers
* Runs the complete test suite
* Cleans up after completion

#### Run Specific Test Files

```bash theme={null}
pytest tests/python/rest_api/test_tasks.py
```

#### Run Specific Test Functions

```bash theme={null}
pytest tests/python/rest_api/test_tasks.py::test_create_task
```

#### Run with Verbose Output

```bash theme={null}
pytest tests/python/ -v
```

#### Run Tests in Parallel

```bash theme={null}
pytest tests/python/ -n auto
```

### SDK Tests

```bash theme={null}
pytest tests/python/sdk/
```

### CLI Tests

```bash theme={null}
pytest tests/python/cli/
```

### Keep Test Services Running

To keep the test environment running for debugging:

```bash theme={null}
pytest tests/python/ --start-services
```

This starts the containers but doesn't run tests, allowing you to:

* Access the test instance at [http://localhost:8080](http://localhost:8080)
* Inspect the database
* Test API endpoints manually

Stop the services:

```bash theme={null}
pytest tests/python/ --stop-services
```

## Running Frontend Tests

### Unit Tests

CVAT uses Jest for frontend unit testing:

```bash theme={null}
cd cvat-ui
yarn test
```

Run with coverage:

```bash theme={null}
yarn test --coverage
```

Run specific test files:

```bash theme={null}
yarn test TaskItem.test.tsx
```

Run in watch mode:

```bash theme={null}
yarn test --watch
```

## Running E2E Tests

E2E tests use Cypress to test the complete application flow.

### Interactive Mode

Open Cypress Test Runner:

```bash theme={null}
cd tests
yarn run cypress:open
```

This opens an interactive interface where you can:

* Select specific tests to run
* Watch tests execute in real-time
* Debug failing tests
* See screenshots and videos

### Headless Mode

Run all E2E tests in headless mode:

```bash theme={null}
cd tests
yarn run cypress:run
```

Run specific test file:

```bash theme={null}
yarn run cypress:run --spec "cypress/e2e/actions_tasks.js"
```

### E2E Test Configuration

Cypress configuration is in `tests/cypress.config.js`. Common settings:

```javascript theme={null}
{
  baseUrl: 'http://localhost:8080',
  video: true,
  screenshotOnRunFailure: true,
  defaultCommandTimeout: 10000
}
```

## Writing Tests

### Writing Python REST API Tests

REST API tests are located in `tests/python/rest_api/`.

#### Test Structure

```python theme={null}
import pytest
from http import HTTPStatus

class TestTasks:
    def test_create_task(self, admin_user):
        """Test creating a new task."""
        # Arrange
        task_spec = {
            "name": "Test Task",
            "labels": [{"name": "car"}]
        }
        
        # Act
        response = admin_user.post(
            "/api/tasks",
            json=task_spec
        )
        
        # Assert
        assert response.status_code == HTTPStatus.CREATED
        assert response.json()["name"] == "Test Task"
        assert len(response.json()["labels"]) == 1

    def test_get_task(self, admin_user, tasks):
        """Test retrieving a task by ID."""
        task_id = tasks[0]["id"]
        
        response = admin_user.get(f"/api/tasks/{task_id}")
        
        assert response.status_code == HTTPStatus.OK
        assert response.json()["id"] == task_id
```

#### Using Fixtures

Common fixtures are defined in `tests/python/shared/fixtures/`:

```python theme={null}
@pytest.fixture
def admin_user():
    """Fixture providing an authenticated admin user."""
    # Returns a test client authenticated as admin
    pass

@pytest.fixture
def tasks():
    """Fixture providing test tasks."""
    # Returns list of test tasks
    pass
```

### Writing Frontend Unit Tests

Frontend tests use Jest and React Testing Library.

#### Component Test Example

```typescript theme={null}
import { render, screen, fireEvent } from '@testing-library/react';
import TaskItem from './TaskItem';

describe('<TaskItem />', () => {
    const mockTask = {
        id: 1,
        name: 'Test Task',
        status: 'annotation',
    };
    
    const mockOnSelect = jest.fn();
    
    beforeEach(() => {
        mockOnSelect.mockClear();
    });
    
    it('renders task name', () => {
        render(<TaskItem task={mockTask} onSelect={mockOnSelect} />);
        expect(screen.getByText('Test Task')).toBeInTheDocument();
    });
    
    it('calls onSelect when clicked', () => {
        render(<TaskItem task={mockTask} onSelect={mockOnSelect} />);
        fireEvent.click(screen.getByText('Test Task'));
        expect(mockOnSelect).toHaveBeenCalledWith(1);
    });
    
    it('displays task status', () => {
        render(<TaskItem task={mockTask} onSelect={mockOnSelect} />);
        expect(screen.getByText('annotation')).toBeInTheDocument();
    });
});
```

### Writing E2E Tests

E2E tests are in `tests/cypress/e2e/`.

#### Cypress Test Example

```javascript theme={null}
describe('Task creation', () => {
    beforeEach(() => {
        cy.visit('/');
        cy.login('admin1', 'password');
    });
    
    it('creates a new task', () => {
        // Navigate to task creation
        cy.contains('Create new task').click();
        
        // Fill in task details
        cy.get('[placeholder="Task name"]').type('My Test Task');
        cy.contains('Add label').click();
        cy.get('[placeholder="Label name"]').type('car');
        
        // Upload images
        cy.get('input[type="file"]').attachFile('sample_image.jpg');
        
        // Submit
        cy.contains('Submit').click();
        
        // Verify task was created
        cy.contains('Task has been created').should('be.visible');
        cy.contains('My Test Task').should('be.visible');
    });
});
```

## Test Database

The test infrastructure uses a pre-populated database.

### Database Reset

The database is automatically restored after each test function to ensure test isolation.

### Updating Test Database

If you need to update the test database:

1. Start test services:
   ```bash theme={null}
   pytest tests/python/ --start-services
   ```

2. Make changes through the UI or API

3. Backup the database:
   ```bash theme={null}
   cd tests/python
   docker exec test_cvat_server_1 python manage.py dumpdata --indent 2 --natural-foreign \
       --exclude=admin --exclude=auth.permission --exclude=authtoken --exclude=contenttypes \
       --exclude=django_rq --exclude=sessions \
       > shared/assets/cvat_db/data.json
   ```

4. Backup data volume:
   ```bash theme={null}
   docker exec test_cvat_server_1 tar --exclude "/home/django/data/cache" -cjv /home/django/data \
       > shared/assets/cvat_db/cvat_data.tar.bz2
   ```

5. Update JSON assets:
   ```bash theme={null}
   python shared/utils/dump_objects.py
   ```

## Test Coverage

Check test coverage for Python:

```bash theme={null}
pytest tests/python/ --cov=cvat --cov-report=html
```

View coverage report:

```bash theme={null}
open htmlcov/index.html
```

For frontend coverage:

```bash theme={null}
cd cvat-ui
yarn test --coverage
```

## Debugging Tests

### Python Tests

Add breakpoints:

```python theme={null}
import pdb; pdb.set_trace()
```

Run with debug output:

```bash theme={null}
pytest tests/python/ -vv -s
```

### Frontend Tests

Run in watch mode:

```bash theme={null}
yarn test --watch
```

Use `console.log` or `debugger` statements.

### Cypress Tests

Use interactive mode:

```bash theme={null}
yarn run cypress:open
```

Add Cypress debug commands:

```javascript theme={null}
cy.debug();
cy.pause();
```

## Continuous Integration

CVAT's CI automatically runs:

* All Python tests
* Frontend unit tests
* E2E tests
* Linters and type checks
* Security scans

Tests must pass before PRs can be merged.

## Troubleshooting

### Port Conflicts

If test containers fail to start due to port conflicts:

```bash theme={null}
docker ps -a
docker stop $(docker ps -a -q)
```

### Database Issues

Recreate the test database:

```bash theme={null}
docker exec test_cvat_db_1 dropdb --if-exists cvat
docker exec test_cvat_db_1 createdb cvat
docker exec test_cvat_server_1 python manage.py migrate
```

### Test Failures

For date/time related failures, update JSON assets:

```bash theme={null}
python tests/python/shared/utils/dump_objects.py
```

## Best Practices

1. **Write isolated tests**: Each test should be independent
2. **Use descriptive names**: Test names should clearly describe what they test
3. **Follow AAA pattern**: Arrange, Act, Assert
4. **Test edge cases**: Include tests for error conditions
5. **Keep tests fast**: Mock external dependencies when possible
6. **Maintain test data**: Keep test fixtures up to date
7. **Clean up**: Ensure tests don't leave side effects

## Next Steps

* Review the [architecture documentation](/contributing/architecture)
* Learn about the [pull request process](/contributing/pull-requests)
* Check the [code style guidelines](/contributing/code-style)