# Inference

## Models

Types:

```python
from nemo_platform.types.inference import OpenAIListModelsResp, OpenAIModelResp
```

Methods:

- client.inference.models.list(\*, workspace) -> OpenAIListModelsResp
- client.inference.models.get(name, \*, workspace) -> OpenAIModelResp

## DeploymentConfigs

Types:

```python
from nemo_platform.types.inference import (
    CreateModelDeploymentConfigRequest,
    K8sNIMOperatorConfig,
    ModelDeploymentConfig,
    ModelDeploymentConfigFilter,
    ModelDeploymentConfigsPage,
    ModelType,
    NIMDeployment,
    UpdateModelDeploymentConfigRequest,
)
```

Methods:

- client.inference.deployment_configs.create(\*, workspace, \*\*params) -> ModelDeploymentConfig
- client.inference.deployment_configs.retrieve(name, \*, workspace) -> ModelDeploymentConfig
- client.inference.deployment_configs.update(name, \*, workspace, \*\*params) -> ModelDeploymentConfig
- client.inference.deployment_configs.list(\*, workspace, \*\*params) -> SyncDefaultPagination[ModelDeploymentConfig]
- client.inference.deployment_configs.delete(name, \*, workspace) -> None

### Versions

Types:

```python
from nemo_platform.types.inference.deployment_configs import VersionListResponse
```

Methods:

- client.inference.deployment_configs.versions.retrieve(name, \*, workspace, config) -> ModelDeploymentConfig
- client.inference.deployment_configs.versions.list(name, \*, workspace) -> VersionListResponse
- client.inference.deployment_configs.versions.delete(name, \*, workspace, config) -> None

## Deployments

Types:

```python
from nemo_platform.types.inference import (
    CreateModelDeploymentRequest,
    ModelDeployment,
    ModelDeploymentFilter,
    ModelDeploymentStatus,
    ModelDeploymentStatusHistoryItem,
    ModelDeploymentsPage,
    UpdateModelDeploymentRequest,
    UpdateModelDeploymentStatusRequest,
    DeploymentListModelsResponse,
)
```

Methods:

- client.inference.deployments.create(\*, workspace, \*\*params) -> ModelDeployment
- client.inference.deployments.retrieve(name, \*, workspace) -> ModelDeployment
- client.inference.deployments.update(name, \*, workspace, \*\*params) -> ModelDeployment
- client.inference.deployments.list(\*, workspace, \*\*params) -> SyncDefaultPagination[ModelDeployment]
- client.inference.deployments.delete(name, \*, workspace) -> object
- client.inference.deployments.list_models(name, \*, workspace) -> DeploymentListModelsResponse
- client.inference.deployments.update_status(name, \*, workspace, \*\*params) -> ModelDeployment

### Versions

Types:

```python
from nemo_platform.types.inference.deployments import VersionListResponse
```

Methods:

- client.inference.deployments.versions.retrieve(name, \*, workspace, deployment) -> ModelDeployment
- client.inference.deployments.versions.list(name, \*, workspace) -> VersionListResponse
- client.inference.deployments.versions.delete(name, \*, workspace, deployment) -> object

## Providers

Types:

```python
from nemo_platform.types.inference import (
    CreateModelProviderRequest,
    ModelProvider,
    ModelProviderFilter,
    ModelProviderSort,
    ModelProviderStatus,
    ModelProvidersPage,
    ServedModelMapping,
    UpdateModelProviderStatusRequest,
    UpsertModelProviderRequest,
)
```

Methods:

- client.inference.providers.create(\*, workspace, \*\*params) -> ModelProvider
- client.inference.providers.retrieve(name, \*, workspace) -> ModelProvider
- client.inference.providers.update(name, \*, workspace, \*\*params) -> ModelProvider
- client.inference.providers.list(\*, workspace, \*\*params) -> SyncDefaultPagination[ModelProvider]
- client.inference.providers.delete(name, \*, workspace) -> None
- client.inference.providers.update_status(name, \*, workspace, \*\*params) -> ModelProvider

## Gateway

### OpenAI

Types:

```python
from nemo_platform.types.inference.gateway import (
    OpenAIPatchResponse,
    OpenAIPostResponse,
    OpenAIPutResponse,
)
```

Methods:

- client.inference.gateway.openai.delete(trailing_uri, \*, workspace) -> object
- client.inference.gateway.openai.get(trailing_uri, \*, workspace) -> object
- client.inference.gateway.openai.patch(trailing_uri, \*, workspace, \*\*params) -> OpenAIPatchResponse
- client.inference.gateway.openai.post(trailing_uri, \*, workspace, \*\*params) -> OpenAIPostResponse
- client.inference.gateway.openai.put(trailing_uri, \*, workspace, \*\*params) -> OpenAIPutResponse

#### V1

##### Models

Types:

```python
from nemo_platform.types.inference.gateway.openai.v1 import OpenAIListModelsResp, OpenAIModelResp
```

Methods:

- client.inference.gateway.openai.v1.models.list(\*, workspace) -> OpenAIListModelsResp
- client.inference.gateway.openai.v1.models.get(name, \*, workspace) -> OpenAIModelResp

### Model

Types:

```python
from nemo_platform.types.inference.gateway import (
    ModelPatchResponse,
    ModelPostResponse,
    ModelPutResponse,
)
```

Methods:

- client.inference.gateway.model.delete(trailing_uri, \*, workspace, name) -> object
- client.inference.gateway.model.get(trailing_uri, \*, workspace, name) -> object
- client.inference.gateway.model.patch(trailing_uri, \*, workspace, name, \*\*params) -> ModelPatchResponse
- client.inference.gateway.model.post(trailing_uri, \*, workspace, name, \*\*params) -> ModelPostResponse
- client.inference.gateway.model.put(trailing_uri, \*, workspace, name, \*\*params) -> ModelPutResponse

### Provider

Types:

```python
from nemo_platform.types.inference.gateway import (
    ProviderPatchResponse,
    ProviderPostResponse,
    ProviderPutResponse,
    ProviderReadyResponse,
)
```

Methods:

- client.inference.gateway.provider.delete(trailing_uri, \*, workspace, name) -> object
- client.inference.gateway.provider.get(trailing_uri, \*, workspace, name) -> object
- client.inference.gateway.provider.patch(trailing_uri, \*, workspace, name, \*\*params) -> ProviderPatchResponse
- client.inference.gateway.provider.post(trailing_uri, \*, workspace, name, \*\*params) -> ProviderPostResponse
- client.inference.gateway.provider.put(trailing_uri, \*, workspace, name, \*\*params) -> ProviderPutResponse
- client.inference.gateway.provider.ready(name, \*, workspace) -> ProviderReadyResponse
