> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/cvat-ai/cvat/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploying Serverless Functions

> Deploy and configure serverless functions using Nuclio for CVAT

## Prerequisites

Before deploying serverless functions, ensure you have:

* CVAT self-hosted installation running
* Docker installed and running
* [Nuclio](https://nuclio.io/) CLI (`nuctl`) installed
* Access to the CVAT serverless functions repository
* Sufficient system resources (4GB+ RAM recommended for model loading)

## Installation

### Install Nuclio CLI

```bash theme={null}
curl -s https://api.github.com/repos/nuclio/nuclio/releases/latest \
  | grep -i "browser_download_url.*nuctl.*$(uname)" \
  | cut -d : -f 2,3 \
  | tr -d \" \
  | wget -O nuctl -qi - && chmod +x nuctl

sudo mv nuctl /usr/local/bin/
```

### Clone Serverless Functions

The serverless functions are included in the CVAT repository:

```bash theme={null}
git clone https://github.com/cvat-ai/cvat.git
cd cvat/serverless
```

## Deployment Options

### Option 1: Deploy All CPU Functions

Deploy all available functions optimized for CPU:

```bash theme={null}
./deploy_cpu.sh
```

This script:

1. Builds the OpenVINO base image
2. Creates a `cvat` project in Nuclio
3. Discovers all `function.yaml` files
4. Builds Docker images for each function
5. Deploys functions to the local Nuclio platform

### Option 2: Deploy All GPU Functions

Deploy GPU-optimized versions (requires NVIDIA GPU and drivers):

```bash theme={null}
./deploy_gpu.sh
```

This uses `function-gpu.yaml` files instead of standard `function.yaml` files.

### Option 3: Deploy Individual Functions

Deploy specific functions manually:

```bash theme={null}
# Create Nuclio project
nuctl create project cvat --platform local

# Build base image (for OpenVINO functions)
docker build -t cvat.openvino.base ./openvino/base

# Deploy a specific function
nuctl deploy --project-name cvat \
  --path ./pytorch/facebookresearch/sam/nuclio \
  --file ./pytorch/facebookresearch/sam/nuclio/function.yaml \
  --platform local
```

## Configuration

### Function YAML Structure

Each function is defined by a `function.yaml` file:

```yaml theme={null}
metadata:
  name: pth-facebookresearch-sam-vit-h
  namespace: cvat
  annotations:
    name: Segment Anything
    version: 2
    type: interactor
    spec: |
      [
        { "name": "object", "type": "mask" }
      ]
    min_pos_points: 0
    min_neg_points: 0
    startswith_box_optional: true

spec:
  description: Interactive object segmentation with Segment-Anything
  runtime: 'python:3.10'
  handler: main:handler
  eventTimeout: 30s
  
  build:
    image: cvat.pth.facebookresearch.sam.vit_h
    baseImage: ubuntu:22.04
    directives:
      preCopy:
        - kind: RUN
          value: apt-get update && apt-get install -y python3-pip
        - kind: RUN
          value: pip install torch torchvision segment-anything

  triggers:
    myHttpTrigger:
      numWorkers: 2
      kind: 'http'
      attributes:
        maxRequestBodySize: 33554432 # 32MB
```

### Key Configuration Parameters

#### Metadata Annotations

* `name`: Display name in CVAT UI
* `type`: Function type (detector, interactor, tracker, reid)
* `spec`: Label schema with supported output types
* `version`: Function version for compatibility

#### Spec Settings

* `runtime`: Python version for the function
* `handler`: Entry point (typically `main:handler`)
* `eventTimeout`: Maximum execution time per request
* `numWorkers`: Number of concurrent workers
* `maxRequestBodySize`: Maximum request payload size

### Environment Variables

Configure CVAT connection when deploying:

```bash theme={null}
nuctl deploy --project-name cvat \
  --path ./function/path \
  --file ./function/path/function.yaml \
  --platform local \
  --env CVAT_FUNCTIONS_REDIS_HOST=cvat_redis_ondisk \
  --env CVAT_FUNCTIONS_REDIS_PORT=6666 \
  --platform-config '{"attributes": {"network": "cvat_cvat"}}'
```

## Connecting to CVAT

### Configure Nuclio Settings

In your CVAT deployment, configure Nuclio connection in `docker-compose.yml`:

```yaml theme={null}
services:
  cvat_server:
    environment:
      CVAT_SERVERLESS: 1
      NUCLIO_SCHEME: http
      NUCLIO_HOST: nuclio
      NUCLIO_PORT: 8070
      NUCLIO_FUNCTION_NAMESPACE: nuclio
      NUCLIO_DEFAULT_TIMEOUT: 30
      NUCLIO_INVOKE_METHOD: direct  # or 'dashboard'
```

### Invoke Methods

**Direct Invocation** (default):

* Calls functions directly via HTTP port
* Lower latency, fewer network hops
* Requires functions accessible from CVAT container

**Dashboard Invocation**:

* Routes through Nuclio dashboard API
* Better for complex networking scenarios
* Slightly higher latency

### Network Configuration

Ensure CVAT and Nuclio containers share a network:

```bash theme={null}
docker network create cvat_cvat

# When deploying functions, specify network:
nuctl deploy --project-name cvat \
  --platform-config '{"attributes": {"network": "cvat_cvat"}}' \
  ...
```

## Verification

### List Deployed Functions

```bash theme={null}
nuctl get function --platform local
```

Expected output:

```
NAMESPACE | NAME                              | PROJECT | STATE | REPLICAS
nuclio    | pth-facebookresearch-sam-vit-h    | cvat    | ready | 2/2
nuclio    | onnx-wongkinyiu-yolov7            | cvat    | ready | 2/2
...
```

### Test Function Invocation

Test a function directly:

```bash theme={null}
echo '{"image": "base64_encoded_image_data"}' | \
  nuctl invoke pth-facebookresearch-sam-vit-h --method POST
```

### Check CVAT Integration

1. Open CVAT UI
2. Create or open a task
3. Navigate to annotation view
4. Check "Tools" menu for available AI models
5. Verify functions appear under "Magic" or "Automatic annotation"

## Troubleshooting

### Function Not Appearing in CVAT

**Check Nuclio function status:**

```bash theme={null}
nuctl get function --platform local
```

**Verify CVAT environment variables:**

```bash theme={null}
docker exec cvat_server env | grep NUCLIO
```

**Check CVAT logs:**

```bash theme={null}
docker logs cvat_server | grep lambda
```

### Function Build Failures

**Increase Docker memory:**

* Some models (especially Mask R-CNN) require 4GB+ RAM
* Adjust Docker Desktop settings or system resources

**Check base image:**

```bash theme={null}
docker images | grep cvat.openvino.base
```

**Rebuild base image:**

```bash theme={null}
docker build -t cvat.openvino.base ./openvino/base
```

### Function Timeout Issues

**Increase timeout in function.yaml:**

```yaml theme={null}
spec:
  eventTimeout: 60s  # Increase from 30s
```

**Update CVAT timeout:**

```yaml theme={null}
environment:
  NUCLIO_DEFAULT_TIMEOUT: 60
```

### Network Connection Errors

**Verify network:**

```bash theme={null}
docker network inspect cvat_cvat
```

**Check function accessibility:**

```bash theme={null}
docker exec cvat_server curl http://nuclio:8070/api/functions
```

**Update invoke method:**

```yaml theme={null}
environment:
  NUCLIO_INVOKE_METHOD: dashboard  # Try alternative method
```

## Resource Management

### Memory Requirements

Typical memory usage per function:

* YOLO v7 (ONNX): \~500MB
* Mask R-CNN (OpenVINO): \~2GB
* SAM (PyTorch): \~2.5GB
* Detectron2 models: \~1-3GB

### GPU Support

For GPU acceleration:

1. **Install NVIDIA Container Toolkit:**

```bash theme={null}
sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker
```

2. **Deploy GPU functions:**

```bash theme={null}
./deploy_gpu.sh
```

3. **Verify GPU access:**

```bash theme={null}
docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
```

## Updating Functions

### Update Existing Function

```bash theme={null}
# Delete old function
nuctl delete function pth-facebookresearch-sam-vit-h --platform local

# Redeploy with updated configuration
nuctl deploy --project-name cvat \
  --path ./pytorch/facebookresearch/sam/nuclio \
  --file ./pytorch/facebookresearch/sam/nuclio/function.yaml \
  --platform local
```

### Update All Functions

```bash theme={null}
# Remove all functions
nuctl delete project cvat --platform local

# Redeploy
./deploy_cpu.sh
```

## Best Practices

1. **Start Small**: Deploy only the functions you need initially
2. **Monitor Resources**: Track memory and CPU usage during operation
3. **Use CPU for Batch**: CPU functions work well for background annotation jobs
4. **Use GPU for Interactive**: GPU accelerates real-time interactor tools
5. **Version Control**: Track function.yaml changes for reproducibility
6. **Test Locally**: Verify functions work before deploying to production
7. **Network Isolation**: Use dedicated networks for security
8. **Regular Updates**: Keep base images and dependencies updated

## Next Steps

<CardGroup cols={2}>
  <Card title="Custom Models" icon="code" href="/self-hosted/serverless/custom-models">
    Create your own serverless functions
  </Card>

  <Card title="Overview" icon="book" href="/self-hosted/serverless/overview">
    Learn more about serverless function types
  </Card>
</CardGroup>