kfp.containers package
- class kfp.containers.ContainerBuilder(gcs_staging=None, default_image_name=None, namespace=None, service_account='kubeflow-pipelines-container-builder', kaniko_executor_image='gcr.io/kaniko-project/executor@sha256:78d44ec4e9cb5545d7f85c1924695c89503ded86a59f92c7ae658afa3cff5400', k8s_client_configuration=None)[source]
Bases:
object
ContainerBuilder helps build a container image.
- build(local_dir, docker_filename: str = 'Dockerfile', target_image=None, timeout=1000)[source]
- Parameters
local_dir (str) – local directory that stores all the necessary build files
docker_filename (str) – the path of the Dockerfile relative to the local_dir
target_image (str) – The container image name where the data will be pushed. Can include tag. If not specified, the function will use the default_image_name specified when creating ContainerBuilder.
timeout (int) – time out in seconds. Default: 1000
- kfp.containers.build_image_from_working_dir(image_name: Optional[str] = None, working_dir: Optional[str] = None, file_filter_re: str = '.*\\.py', timeout: int = 1000, base_image: Optional[str] = None, builder: Optional[kfp.containers._container_builder.ContainerBuilder] = None) str [source]
Builds and pushes a new container image that captures the current python working directory.
This function recursively scans the working directory and captures the following files in the container image context:
requirements.txt
filesAll python files (can be overridden by passing a different file_filter_re argument)
The function generates Dockerfile that starts from a python container image, install packages from requirements.txt (if present) and copies all the captured python files to the container image. The Dockerfile can be overridden by placing a custom Dockerfile in the root of the working directory.
- Parameters
image_name – Optional. The image repo name where the new container image will be pushed. The name will be generated if not not set.
working_dir – Optional. The directory that will be captured. The current directory will be used if omitted.
file_filter_re – Optional. A regular expression that will be used to decide which files to include in the container building context.
timeout – Optional. The image building timeout in seconds.
base_image – Optional. The container image to use as the base for the new image. If not set, the Google Deep Learning Tensorflow CPU image will be used.
builder –
Optional. An instance of
kfp.containers.ContainerBuilder
or compatible class that will be used to build the image. The default builder uses “kubeflow-pipelines-container-builder” service account in “kubeflow” namespace. It works with Kubeflow Pipelines clusters installed in “kubeflow” namespace using Google Cloud Marketplace or Standalone with version > 0.4.0. If your Kubeflow Pipelines is installed in a different namespace, you should useContainerBuilder(namespace='<your-kfp-namespace>', ...)
.Depending on how you installed Kubeflow Pipelines, you need to configure your
ContainerBuilder
instance’s namespace and service_account:For clusters installed with Kubeflow >= 0.7, use
ContainerBuilder(namespace='<your-user-namespace>', service_account='default-editor', ...)
. You can omit the namespace if you use kfp sdk from in-cluster notebook, it uses notebook namespace by default.For clusters installed with Kubeflow < 0.7, use
ContainerBuilder(service_account='default', ...)
.For clusters installed using Google Cloud Marketplace or Standalone with version <= 0.4.0, use
ContainerBuilder(namespace='<your-kfp-namespace>' service_account='default')
You may refer to installation guide for more details about different installation options.
- Returns
The full name of the container image including the hash digest. E.g.
gcr.io/my-org/my-image@sha256:86c1...793c
.