经验首页 前端设计 程序设计 Java相关 移动开发 数据库/运维 软件/图像 大数据/云计算 其他经验
当前位置:技术经验 » 数据库/运维 » Kubernetes » 查看文章
Kubernetes容器生命周期 —— 钩子函数详解(postStart、preStop)
来源:cnblogs  作者:人艰不拆_zmc  时间:2024/6/25 8:51:25  对本文有异议

1、概述

  容器生命周期钩子(Container Lifecycle Hooks)监听容器生命周期的特定事件,并在事件发生时执行已注册的回调函数。

  钩子函数能够感知自身生命周期中的事件,并在相应的时刻到来时运行用户指定的程序代码。

  kubernetes在主容器的启动之后和停止之前提供了两个钩子函数:

  1. postStart:一种容器钩子。该钩子在容器被创建后立刻触发,通知容器它已经被创建。该钩子不需要向其所对应的Hook Handler传入任何参数。如果该钩子对应的Hook Handler执行失败,则该容器会终止运行,并根据该容器的重启策略决定是否要重启该容器。更多信息,请参见Container Lifecycle Hooks
  2. preStop :一种容器钩子。该钩子在容器被删除前触发,其所对应的Hook Handler必须在删除该容器的请求发送给Docker Daemon之前完成。在该钩子对应的Hook Handler完成后不论执行的结果如何,Docker Daemon会发送一个SIGTERN信号给Docker Daemon来删除该容器。更多信息,请参见Container Lifecycle Hooks 以及《详解Kubernetes Pod优雅退出》这篇博文。

2、钩子函数定义方式

  钩子的回调函数支持两种方式定义动作,下面以 postStart 为例讲解,preStop 定义方式与其一致:

2.1 exec模式

在容器中执行指定的命令。如果命令退出时返回码为0,判定为执行成功。

示例:

  1. lifecycle:
  2. postStart:
  3. exec:
  4. command:
  5. - cat
  6. - /var/lib/redis.conf

2.2 httpGet模式

对容器中的指定端点执行HTTP GET请求,如果响应的状态码大于等于200且小于400,判定为执行成功。

httpGet模式支持以下配置参数:

  • Host:HTTP请求主机地址,不设置时默认为Pod的IP地址。

  • Path:HTTP请求路径,默认值是/。

  • Port:HTTP请求端口号。

  • Scheme:协议类型,支持HTTP和HTTPS协议,默认是HTTP。

  • HttpHeaders:自定义HTTP请求头。

示例:访问 http://192.168.2.150:80/users

  1. lifecycle:
  2. postStart:
  3. httpGet:
  4. path: /users
  5. port: 80
  6. host: 192.168.2.150
  7. scheme: HTTP # 或者HTTPS

3、源码讲解

3.1 PostStart

Kubernetes源码(1.21.5,已在源码中移除dockershim模块):

 pkg/kubelet/kuberuntime/kuberuntime_container.go:

开始容器方法主要包括拉取镜像、创建容器、start容器、运行容器postStart钩子函数。

  1. // startContainer starts a container and returns a message indicates why it is failed on error.
  2. // It starts the container through the following steps:
  3. // * pull the image
  4. // * create the container
  5. // * start the container
  6. // * run the post start lifecycle hooks (if applicable)
  7. func (m *kubeGenericRuntimeManager) startContainer(podSandboxID string, podSandboxConfig *runtimeapi.PodSandboxConfig, spec *startSpec, pod *v1.Pod, podStatus *kubecontainer.PodStatus, pullSecrets []v1.Secret, podIP string, podIPs []string) (string, error) {
  8. container := spec.container
  9.  
  10. // Step 1: pull the image.
  11. imageRef, msg, err := m.imagePuller.EnsureImageExists(pod, container, pullSecrets, podSandboxConfig)
  12. if err != nil {
  13. s, _ := grpcstatus.FromError(err)
  14. m.recordContainerEvent(pod, container, "", v1.EventTypeWarning, events.FailedToCreateContainer, "Error: %v", s.Message())
  15. return msg, err
  16. }
  17.  
  18. // Step 2: create the container.
  19. // For a new container, the RestartCount should be 0
  20. restartCount := 0
  21. containerStatus := podStatus.FindContainerStatusByName(container.Name)
  22. if containerStatus != nil {
  23. restartCount = containerStatus.RestartCount + 1
  24. }
  25.  
  26. target, err := spec.getTargetID(podStatus)
  27. if err != nil {
  28. s, _ := grpcstatus.FromError(err)
  29. m.recordContainerEvent(pod, container, "", v1.EventTypeWarning, events.FailedToCreateContainer, "Error: %v", s.Message())
  30. return s.Message(), ErrCreateContainerConfig
  31. }
  32.  
  33. containerConfig, cleanupAction, err := m.generateContainerConfig(container, pod, restartCount, podIP, imageRef, podIPs, target)
  34. if cleanupAction != nil {
  35. defer cleanupAction()
  36. }
  37. if err != nil {
  38. s, _ := grpcstatus.FromError(err)
  39. m.recordContainerEvent(pod, container, "", v1.EventTypeWarning, events.FailedToCreateContainer, "Error: %v", s.Message())
  40. return s.Message(), ErrCreateContainerConfig
  41. }
  42.  
  43. err = m.internalLifecycle.PreCreateContainer(pod, container, containerConfig)
  44. if err != nil {
  45. s, _ := grpcstatus.FromError(err)
  46. m.recordContainerEvent(pod, container, "", v1.EventTypeWarning, events.FailedToCreateContainer, "Internal PreCreateContainer hook failed: %v", s.Message())
  47. return s.Message(), ErrPreCreateHook
  48. }
  49.  
  50. containerID, err := m.runtimeService.CreateContainer(podSandboxID, containerConfig, podSandboxConfig)
  51. if err != nil {
  52. s, _ := grpcstatus.FromError(err)
  53. m.recordContainerEvent(pod, container, containerID, v1.EventTypeWarning, events.FailedToCreateContainer, "Error: %v", s.Message())
  54. return s.Message(), ErrCreateContainer
  55. }
  56. err = m.internalLifecycle.PreStartContainer(pod, container, containerID)
  57. if err != nil {
  58. s, _ := grpcstatus.FromError(err)
  59. m.recordContainerEvent(pod, container, containerID, v1.EventTypeWarning, events.FailedToStartContainer, "Internal PreStartContainer hook failed: %v", s.Message())
  60. return s.Message(), ErrPreStartHook
  61. }
  62. m.recordContainerEvent(pod, container, containerID, v1.EventTypeNormal, events.CreatedContainer, fmt.Sprintf("Created container %s", container.Name))
  63.  
  64. // Step 3: start the container.
  65. err = m.runtimeService.StartContainer(containerID)
  66. if err != nil {
  67. s, _ := grpcstatus.FromError(err)
  68. m.recordContainerEvent(pod, container, containerID, v1.EventTypeWarning, events.FailedToStartContainer, "Error: %v", s.Message())
  69. return s.Message(), kubecontainer.ErrRunContainer
  70. }
  71. m.recordContainerEvent(pod, container, containerID, v1.EventTypeNormal, events.StartedContainer, fmt.Sprintf("Started container %s", container.Name))
  72.  
  73. // Symlink container logs to the legacy container log location for cluster logging
  74. // support.
  75. // TODO(random-liu): Remove this after cluster logging supports CRI container log path.
  76. containerMeta := containerConfig.GetMetadata()
  77. sandboxMeta := podSandboxConfig.GetMetadata()
  78. legacySymlink := legacyLogSymlink(containerID, containerMeta.Name, sandboxMeta.Name,
  79. sandboxMeta.Namespace)
  80. containerLog := filepath.Join(podSandboxConfig.LogDirectory, containerConfig.LogPath)
  81. // only create legacy symlink if containerLog path exists (or the error is not IsNotExist).
  82. // Because if containerLog path does not exist, only dangling legacySymlink is created.
  83. // This dangling legacySymlink is later removed by container gc, so it does not make sense
  84. // to create it in the first place. it happens when journald logging driver is used with docker.
  85. if _, err := m.osInterface.Stat(containerLog); !os.IsNotExist(err) {
  86. if err := m.osInterface.Symlink(containerLog, legacySymlink); err != nil {
  87. klog.ErrorS(err, "Failed to create legacy symbolic link", "path", legacySymlink,
  88. "containerID", containerID, "containerLogPath", containerLog)
  89. }
  90. }
  91.  
  92. // Step 4: execute the post start hook.
  93. if container.Lifecycle != nil && container.Lifecycle.PostStart != nil {
  94. kubeContainerID := kubecontainer.ContainerID{
  95. Type: m.runtimeName,
  96. ID: containerID,
  97. }
  98. msg, handlerErr := m.runner.Run(kubeContainerID, pod, container, container.Lifecycle.PostStart)
  99. if handlerErr != nil {
  100. m.recordContainerEvent(pod, container, kubeContainerID.ID, v1.EventTypeWarning, events.FailedPostStartHook, msg)
  101. if err := m.killContainer(pod, kubeContainerID, container.Name, "FailedPostStartHook", reasonFailedPostStartHook, nil); err != nil {
  102. klog.ErrorS(fmt.Errorf("%s: %v", ErrPostStartHook, handlerErr), "Failed to kill container", "pod", klog.KObj(pod),
  103. "podUID", pod.UID, "containerName", container.Name, "containerID", kubeContainerID.String())
  104. }
  105. return msg, fmt.Errorf("%s: %v", ErrPostStartHook, handlerErr)
  106. }
  107. }
  108.  
  109. return "", nil
  110. }

(1)关注下创建容器,kubelet调用docker csi,相当于执行 docker create 命令:

pkg/kubelet/cri/remote/remote_runtime.go:

  1. // CreateContainer creates a new container in the specified PodSandbox.
  2. func (r *remoteRuntimeService) CreateContainer(podSandBoxID string, config *runtimeapi.ContainerConfig, sandboxConfig *runtimeapi.PodSandboxConfig) (string, error) {
  3. klog.V(10).InfoS("[RemoteRuntimeService] CreateContainer", "podSandboxID", podSandBoxID, "timeout", r.timeout)
  4. ctx, cancel := getContextWithTimeout(r.timeout)
  5. defer cancel()
  6.  
  7. resp, err := r.runtimeClient.CreateContainer(ctx, &runtimeapi.CreateContainerRequest{
  8. PodSandboxId: podSandBoxID,
  9. Config: config,
  10. SandboxConfig: sandboxConfig,
  11. })
  12. if err != nil {
  13. klog.ErrorS(err, "CreateContainer in sandbox from runtime service failed", "podSandboxID", podSandBoxID)
  14. return "", err
  15. }
  16.  
  17. klog.V(10).InfoS("[RemoteRuntimeService] CreateContainer", "podSandboxID", podSandBoxID, "containerID", resp.ContainerId)
  18. if resp.ContainerId == "" {
  19. errorMessage := fmt.Sprintf("ContainerId is not set for container %q", config.GetMetadata())
  20. err := errors.New(errorMessage)
  21. klog.ErrorS(err, "CreateContainer failed")
  22. return "", err
  23. }
  24.  
  25. return resp.ContainerId, nil
  26. }

(2)start容器,kubelet调用docker csi,相当于执行 docker start 命令:

pkg/kubelet/cri/remote/remote_runtime.go:

  1. // StartContainer starts the container.
  2. func (r *remoteRuntimeService) StartContainer(containerID string) error {
  3. klog.V(10).InfoS("[RemoteRuntimeService] StartContainer", "containerID", containerID, "timeout", r.timeout)
  4. ctx, cancel := getContextWithTimeout(r.timeout)
  5. defer cancel()
  6.  
  7. _, err := r.runtimeClient.StartContainer(ctx, &runtimeapi.StartContainerRequest{
  8. ContainerId: containerID,
  9. })
  10. if err != nil {
  11. klog.ErrorS(err, "StartContainer from runtime service failed", "containerID", containerID)
  12. return err
  13. }
  14. klog.V(10).InfoS("[RemoteRuntimeService] StartContainer Response", "containerID", containerID)
  15.  
  16. return nil
  17. }

(3)运行容器postStart钩子函数(kubelet):

pkg/kubelet/lifecycle/handlers.go:

  1. func (hr *handlerRunner) Run(containerID kubecontainer.ContainerID, pod *v1.Pod, container *v1.Container, handler *v1.Handler) (string, error) {
  2. switch {
  3. case handler.Exec != nil:
  4. var msg string
  5. // TODO(tallclair): Pass a proper timeout value.
  6. output, err := hr.commandRunner.RunInContainer(containerID, handler.Exec.Command, 0)
  7. if err != nil {
  8. msg = fmt.Sprintf("Exec lifecycle hook (%v) for Container %q in Pod %q failed - error: %v, message: %q", handler.Exec.Command, container.Name, format.Pod(pod), err, string(output))
  9. klog.V(1).ErrorS(err, "Exec lifecycle hook for Container in Pod failed", "execCommand", handler.Exec.Command, "containerName", container.Name, "pod", klog.KObj(pod), "message", string(output))
  10. }
  11. return msg, err
  12. case handler.HTTPGet != nil:
  13. msg, err := hr.runHTTPHandler(pod, container, handler)
  14. if err != nil {
  15. msg = fmt.Sprintf("HTTP lifecycle hook (%s) for Container %q in Pod %q failed - error: %v, message: %q", handler.HTTPGet.Path, container.Name, format.Pod(pod), err, msg)
  16. klog.V(1).ErrorS(err, "HTTP lifecycle hook for Container in Pod failed", "path", handler.HTTPGet.Path, "containerName", container.Name, "pod", klog.KObj(pod))
  17. }
  18. return msg, err
  19. default:
  20. err := fmt.Errorf("invalid handler: %v", handler)
  21. msg := fmt.Sprintf("Cannot run handler: %v", err)
  22. klog.ErrorS(err, "Cannot run handler")
  23. return msg, err
  24. }
  25. }

cri-dockerd源码:

(1)创建容器

core/container_create.go:

  1. // CreateContainer creates a new container in the given PodSandbox
  2. // Docker cannot store the log to an arbitrary location (yet), so we create an
  3. // symlink at LogPath, linking to the actual path of the log.
  4. func (ds *dockerService) CreateContainer(
  5. _ context.Context,
  6. r *v1.CreateContainerRequest,
  7. ) (*v1.CreateContainerResponse, error) {
  8. podSandboxID := r.PodSandboxId
  9. config := r.GetConfig()
  10. sandboxConfig := r.GetSandboxConfig()
  11.  
  12. if config == nil {
  13. return nil, fmt.Errorf("container config is nil")
  14. }
  15. if sandboxConfig == nil {
  16. return nil, fmt.Errorf("sandbox config is nil for container %q", config.Metadata.Name)
  17. }
  18.  
  19. labels := makeLabels(config.GetLabels(), config.GetAnnotations())
  20. // Apply a the container type label.
  21. labels[containerTypeLabelKey] = containerTypeLabelContainer
  22. // Write the container log path in the labels.
  23. labels[containerLogPathLabelKey] = filepath.Join(sandboxConfig.LogDirectory, config.LogPath)
  24. // Write the sandbox ID in the labels.
  25. labels[sandboxIDLabelKey] = podSandboxID
  26.  
  27. apiVersion, err := ds.getDockerAPIVersion()
  28. if err != nil {
  29. return nil, fmt.Errorf("unable to get the docker API version: %v", err)
  30. }
  31.  
  32. image := ""
  33. if iSpec := config.GetImage(); iSpec != nil {
  34. image = iSpec.Image
  35. }
  36. containerName := makeContainerName(sandboxConfig, config)
  37. mounts := config.GetMounts()
  38. terminationMessagePath, _ := config.Annotations["io.kubernetes.container.terminationMessagePath"]
  39.  
  40. sandboxInfo, err := ds.client.InspectContainer(r.GetPodSandboxId())
  41. if err != nil {
  42. return nil, fmt.Errorf("unable to get container's sandbox ID: %v", err)
  43. }
  44. createConfig := dockerbackend.ContainerCreateConfig{
  45. Name: containerName,
  46. Config: &container.Config{
  47. Entrypoint: strslice.StrSlice(config.Command),
  48. Cmd: strslice.StrSlice(config.Args),
  49. Env: libdocker.GenerateEnvList(config.GetEnvs()),
  50. Image: image,
  51. WorkingDir: config.WorkingDir,
  52. Labels: labels,
  53. // Interactive containers:
  54. OpenStdin: config.Stdin,
  55. StdinOnce: config.StdinOnce,
  56. Tty: config.Tty,
  57. // Disable Docker's health check until we officially support it
  58. // (https://github.com/kubernetes/kubernetes/issues/25829).
  59. Healthcheck: &container.HealthConfig{
  60. Test: []string{"NONE"},
  61. },
  62. },
  63. HostConfig: &container.HostConfig{
  64. Mounts: libdocker.GenerateMountBindings(mounts, terminationMessagePath),
  65. RestartPolicy: container.RestartPolicy{
  66. Name: "no",
  67. },
  68. Runtime: sandboxInfo.HostConfig.Runtime,
  69. },
  70. }
  71.  
  72. // Only request relabeling if the pod provides an SELinux context. If the pod
  73. // does not provide an SELinux context relabeling will label the volume with
  74. // the container's randomly allocated MCS label. This would restrict access
  75. // to the volume to the container which mounts it first.
  76. if selinuxOpts := config.GetLinux().GetSecurityContext().GetSelinuxOptions(); selinuxOpts != nil {
  77. mountLabel, err := selinuxMountLabel(selinuxOpts)
  78. if err != nil {
  79. return nil, fmt.Errorf("unable to generate SELinux mount label: %v", err)
  80. }
  81. if mountLabel != "" {
  82. // Equates to "Z" in the old bind API
  83. const shared = false
  84. for _, m := range mounts {
  85. if m.SelinuxRelabel {
  86. if err := label.Relabel(m.HostPath, mountLabel, shared); err != nil {
  87. return nil, fmt.Errorf("unable to relabel %q with %q: %v", m.HostPath, mountLabel, err)
  88. }
  89. }
  90. }
  91. }
  92. }
  93.  
  94. hc := createConfig.HostConfig
  95. err = ds.updateCreateConfig(
  96. &createConfig,
  97. config,
  98. sandboxConfig,
  99. podSandboxID,
  100. securityOptSeparator,
  101. apiVersion,
  102. )
  103. if err != nil {
  104. return nil, fmt.Errorf("failed to update container create config: %v", err)
  105. }
  106. // Set devices for container.
  107. devices := make([]container.DeviceMapping, len(config.Devices))
  108. for i, device := range config.Devices {
  109. devices[i] = container.DeviceMapping{
  110. PathOnHost: device.HostPath,
  111. PathInContainer: device.ContainerPath,
  112. CgroupPermissions: device.Permissions,
  113. }
  114. }
  115. hc.Resources.Devices = devices
  116.  
  117. securityOpts, err := ds.getSecurityOpts(
  118. config.GetLinux().GetSecurityContext().GetSeccomp(),
  119. securityOptSeparator,
  120. )
  121. if err != nil {
  122. return nil, fmt.Errorf(
  123. "failed to generate security options for container %q: %v",
  124. config.Metadata.Name,
  125. err,
  126. )
  127. }
  128.  
  129. hc.SecurityOpt = append(hc.SecurityOpt, securityOpts...)
  130.  
  131. cleanupInfo, err := ds.applyPlatformSpecificDockerConfig(r, &createConfig)
  132. if err != nil {
  133. return nil, err
  134. }
  135.  
  136. createResp, createErr := ds.client.CreateContainer(createConfig)
  137. if createErr != nil {
  138. createResp, createErr = recoverFromCreationConflictIfNeeded(
  139. ds.client,
  140. createConfig,
  141. createErr,
  142. )
  143. }
  144.  
  145. if createResp != nil {
  146. containerID := createResp.ID
  147.  
  148. if cleanupInfo != nil {
  149. // we don't perform the clean up just yet at that could destroy information
  150. // needed for the container to start (e.g. Windows credentials stored in
  151. // registry keys); instead, we'll clean up when the container gets removed
  152. ds.setContainerCleanupInfo(containerID, cleanupInfo)
  153. }
  154. return &v1.CreateContainerResponse{ContainerId: containerID}, nil
  155. }
  156.  
  157. // the creation failed, let's clean up right away - we ignore any errors though,
  158. // this is best effort
  159. ds.performPlatformSpecificContainerCleanupAndLogErrors(containerName, cleanupInfo)
  160.  
  161. return nil, createErr
  162. }

 调用CreateContainer方法(libdocker/kube_docker_client.go),然后作为 docker 客户端调用 docker 服务端 ContainerCreate 方法。

  1. func (d *kubeDockerClient) CreateContainer(
  2. opts dockerbackend.ContainerCreateConfig,
  3. ) (*dockercontainer.CreateResponse, error) {
  4. ctx, cancel := context.WithTimeout(context.Background(), d.timeout)
  5. defer cancel()
  6. // we provide an explicit default shm size as to not depend on docker daemon.
  7. if opts.HostConfig != nil && opts.HostConfig.ShmSize <= 0 {
  8. opts.HostConfig.ShmSize = defaultShmSize
  9. }
  10. createResp, err := d.client.ContainerCreate(
  11. ctx,
  12. opts.Config,
  13. opts.HostConfig,
  14. opts.NetworkingConfig,
  15. nil,
  16. opts.Name,
  17. )
  18. if ctxErr := contextError(ctx); ctxErr != nil {
  19. return nil, ctxErr
  20. }
  21. if err != nil {
  22. return nil, err
  23. }
  24. return &createResp, nil
  25. }

vendor/github.com/docker/docker/client/container_create.go:

  1. // ContainerCreate creates a new container based on the given configuration.
  2. // It can be associated with a name, but it's not mandatory.
  3. func (cli *Client) ContainerCreate(ctx context.Context, config *container.Config, hostConfig *container.HostConfig, networkingConfig *network.NetworkingConfig, platform *ocispec.Platform, containerName string) (container.CreateResponse, error) {
  4. var response container.CreateResponse
  5.  
  6. // Make sure we negotiated (if the client is configured to do so),
  7. // as code below contains API-version specific handling of options.
  8. //
  9. // Normally, version-negotiation (if enabled) would not happen until
  10. // the API request is made.
  11. if err := cli.checkVersion(ctx); err != nil {
  12. return response, err
  13. }
  14.  
  15. if err := cli.NewVersionError(ctx, "1.25", "stop timeout"); config != nil && config.StopTimeout != nil && err != nil {
  16. return response, err
  17. }
  18. if err := cli.NewVersionError(ctx, "1.41", "specify container image platform"); platform != nil && err != nil {
  19. return response, err
  20. }
  21. if err := cli.NewVersionError(ctx, "1.44", "specify health-check start interval"); config != nil && config.Healthcheck != nil && config.Healthcheck.StartInterval != 0 && err != nil {
  22. return response, err
  23. }
  24. if err := cli.NewVersionError(ctx, "1.44", "specify mac-address per network"); hasEndpointSpecificMacAddress(networkingConfig) && err != nil {
  25. return response, err
  26. }
  27.  
  28. if hostConfig != nil {
  29. if versions.LessThan(cli.ClientVersion(), "1.25") {
  30. // When using API 1.24 and under, the client is responsible for removing the container
  31. hostConfig.AutoRemove = false
  32. }
  33. if versions.GreaterThanOrEqualTo(cli.ClientVersion(), "1.42") || versions.LessThan(cli.ClientVersion(), "1.40") {
  34. // KernelMemory was added in API 1.40, and deprecated in API 1.42
  35. hostConfig.KernelMemory = 0
  36. }
  37. if platform != nil && platform.OS == "linux" && versions.LessThan(cli.ClientVersion(), "1.42") {
  38. // When using API under 1.42, the Linux daemon doesn't respect the ConsoleSize
  39. hostConfig.ConsoleSize = [2]uint{0, 0}
  40. }
  41. }
  42.  
  43. // Since API 1.44, the container-wide MacAddress is deprecated and will trigger a WARNING if it's specified.
  44. if versions.GreaterThanOrEqualTo(cli.ClientVersion(), "1.44") {
  45. config.MacAddress = "" //nolint:staticcheck // ignore SA1019: field is deprecated, but still used on API < v1.44.
  46. }
  47.  
  48. query := url.Values{}
  49. if p := formatPlatform(platform); p != "" {
  50. query.Set("platform", p)
  51. }
  52.  
  53. if containerName != "" {
  54. query.Set("name", containerName)
  55. }
  56.  
  57. body := configWrapper{
  58. Config: config,
  59. HostConfig: hostConfig,
  60. NetworkingConfig: networkingConfig,
  61. }
  62.  
  63. serverResp, err := cli.post(ctx, "/containers/create", query, body, nil)
  64. defer ensureReaderClosed(serverResp)
  65. if err != nil {
  66. return response, err
  67. }
  68.  
  69. err = json.NewDecoder(serverResp.body).Decode(&response)
  70. return response, err
  71. }

(2)start容器

core/container_start.go

  1. // StartContainer starts the container.
  2. func (ds *dockerService) StartContainer(
  3. _ context.Context,
  4. r *v1.StartContainerRequest,
  5. ) (*v1.StartContainerResponse, error) {
  6. err := ds.client.StartContainer(r.ContainerId)
  7.  
  8. // Create container log symlink for all containers (including failed ones).
  9. if linkError := ds.createContainerLogSymlink(r.ContainerId); linkError != nil {
  10. // Do not stop the container if we failed to create symlink because:
  11. // 1. This is not a critical failure.
  12. // 2. We don't have enough information to properly stop container here.
  13. // Kubelet will surface this error to user via an event.
  14. return nil, linkError
  15. }
  16.  
  17. if err != nil {
  18. err = transformStartContainerError(err)
  19. return nil, fmt.Errorf("failed to start container %q: %v", r.ContainerId, err)
  20. }
  21.  
  22. return &v1.StartContainerResponse{}, nil
  23. }

libdocker/kube_docker_client.go

  1. func (d *kubeDockerClient) StartContainer(id string) error {
  2. ctx, cancel := context.WithTimeout(context.Background(), d.timeout)
  3. defer cancel()
  4. err := d.client.ContainerStart(ctx, id, dockercontainer.StartOptions{})
  5. if ctxErr := contextError(ctx); ctxErr != nil {
  6. return ctxErr
  7. }
  8. return err
  9. }

github.com/docker/docker/client/container_start.go

  1. // ContainerStart sends a request to the docker daemon to start a container.
  2. func (cli *Client) ContainerStart(ctx context.Context, containerID string, options container.StartOptions) error {
  3. query := url.Values{}
  4. if len(options.CheckpointID) != 0 {
  5. query.Set("checkpoint", options.CheckpointID)
  6. }
  7. if len(options.CheckpointDir) != 0 {
  8. query.Set("checkpoint-dir", options.CheckpointDir)
  9. }
  10.  
  11. resp, err := cli.post(ctx, "/containers/"+containerID+"/start", query, nil, nil)
  12. ensureReaderClosed(resp)
  13. return err
  14. }

moby源码(docker改名为moby):

(1)创建容器

api/server/router/container/container.go:

  1. router.NewPostRoute("/containers/create", r.postContainersCreate),

api/server/router/container/container_routes.go:

  1. func (s *containerRouter) postContainersCreate(ctx context.Context, w http.ResponseWriter, r *http.Request, vars map[string]string) error {
  2. if err := httputils.ParseForm(r); err != nil {
  3. return err
  4. }
  5. if err := httputils.CheckForJSON(r); err != nil {
  6. return err
  7. }
  8.  
  9. name := r.Form.Get("name")
  10.  
  11. config, hostConfig, networkingConfig, err := s.decoder.DecodeConfig(r.Body)
  12. if err != nil {
  13. return err
  14. }
  15. version := httputils.VersionFromContext(ctx)
  16. adjustCPUShares := versions.LessThan(version, "1.19")
  17.  
  18. // When using API 1.24 and under, the client is responsible for removing the container
  19. if hostConfig != nil && versions.LessThan(version, "1.25") {
  20. hostConfig.AutoRemove = false
  21. }
  22.  
  23. if hostConfig != nil && versions.LessThan(version, "1.40") {
  24. // Ignore BindOptions.NonRecursive because it was added in API 1.40.
  25. for _, m := range hostConfig.Mounts {
  26. if bo := m.BindOptions; bo != nil {
  27. bo.NonRecursive = false
  28. }
  29. }
  30. // Ignore KernelMemoryTCP because it was added in API 1.40.
  31. hostConfig.KernelMemoryTCP = 0
  32.  
  33. // Older clients (API < 1.40) expects the default to be shareable, make them happy
  34. if hostConfig.IpcMode.IsEmpty() {
  35. hostConfig.IpcMode = container.IPCModeShareable
  36. }
  37. }
  38. if hostConfig != nil && versions.LessThan(version, "1.41") && !s.cgroup2 {
  39. // Older clients expect the default to be "host" on cgroup v1 hosts
  40. if hostConfig.CgroupnsMode.IsEmpty() {
  41. hostConfig.CgroupnsMode = container.CgroupnsModeHost
  42. }
  43. }
  44.  
  45. if hostConfig != nil && versions.LessThan(version, "1.42") {
  46. for _, m := range hostConfig.Mounts {
  47. // Ignore BindOptions.CreateMountpoint because it was added in API 1.42.
  48. if bo := m.BindOptions; bo != nil {
  49. bo.CreateMountpoint = false
  50. }
  51.  
  52. // These combinations are invalid, but weren't validated in API < 1.42.
  53. // We reset them here, so that validation doesn't produce an error.
  54. if o := m.VolumeOptions; o != nil && m.Type != mount.TypeVolume {
  55. m.VolumeOptions = nil
  56. }
  57. if o := m.TmpfsOptions; o != nil && m.Type != mount.TypeTmpfs {
  58. m.TmpfsOptions = nil
  59. }
  60. if bo := m.BindOptions; bo != nil {
  61. // Ignore BindOptions.CreateMountpoint because it was added in API 1.42.
  62. bo.CreateMountpoint = false
  63. }
  64. }
  65. }
  66.  
  67. if hostConfig != nil && versions.GreaterThanOrEqualTo(version, "1.42") {
  68. // Ignore KernelMemory removed in API 1.42.
  69. hostConfig.KernelMemory = 0
  70. for _, m := range hostConfig.Mounts {
  71. if o := m.VolumeOptions; o != nil && m.Type != mount.TypeVolume {
  72. return errdefs.InvalidParameter(fmt.Errorf("VolumeOptions must not be specified on mount type %q", m.Type))
  73. }
  74. if o := m.BindOptions; o != nil && m.Type != mount.TypeBind {
  75. return errdefs.InvalidParameter(fmt.Errorf("BindOptions must not be specified on mount type %q", m.Type))
  76. }
  77. if o := m.TmpfsOptions; o != nil && m.Type != mount.TypeTmpfs {
  78. return errdefs.InvalidParameter(fmt.Errorf("TmpfsOptions must not be specified on mount type %q", m.Type))
  79. }
  80. }
  81. }
  82.  
  83. if hostConfig != nil && runtime.GOOS == "linux" && versions.LessThan(version, "1.42") {
  84. // ConsoleSize is not respected by Linux daemon before API 1.42
  85. hostConfig.ConsoleSize = [2]uint{0, 0}
  86. }
  87.  
  88. var platform *specs.Platform
  89. if versions.GreaterThanOrEqualTo(version, "1.41") {
  90. if v := r.Form.Get("platform"); v != "" {
  91. p, err := platforms.Parse(v)
  92. if err != nil {
  93. return errdefs.InvalidParameter(err)
  94. }
  95. platform = &p
  96. }
  97. }
  98.  
  99. if hostConfig != nil && hostConfig.PidsLimit != nil && *hostConfig.PidsLimit <= 0 {
  100. // Don't set a limit if either no limit was specified, or "unlimited" was
  101. // explicitly set.
  102. // Both `0` and `-1` are accepted as "unlimited", and historically any
  103. // negative value was accepted, so treat those as "unlimited" as well.
  104. hostConfig.PidsLimit = nil
  105. }
  106.  
  107. ccr, err := s.backend.ContainerCreate(types.ContainerCreateConfig{
  108. Name: name,
  109. Config: config,
  110. HostConfig: hostConfig,
  111. NetworkingConfig: networkingConfig,
  112. AdjustCPUShares: adjustCPUShares,
  113. Platform: platform,
  114. })
  115. if err != nil {
  116. return err
  117. }
  118.  
  119. return httputils.WriteJSON(w, http.StatusCreated, ccr)
  120. }

(2)start容器

api/server/router/container/container.go:

  1. router.NewPostRoute("/containers/{name:.*}/start", r.postContainersStart),

daemon/start.go:

  1. // containerStart prepares the container to run by setting up everything the
  2. // container needs, such as storage and networking, as well as links
  3. // between containers. The container is left waiting for a signal to
  4. // begin running.
  5. func (daemon *Daemon) containerStart(container *container.Container, checkpoint string, checkpointDir string, resetRestartManager bool) (err error) {
  6. start := time.Now()
  7. container.Lock()
  8. defer container.Unlock()
  9.  
  10. if resetRestartManager && container.Running { // skip this check if already in restarting step and resetRestartManager==false
  11. return nil
  12. }
  13.  
  14. if container.RemovalInProgress || container.Dead {
  15. return errdefs.Conflict(errors.New("container is marked for removal and cannot be started"))
  16. }
  17.  
  18. if checkpointDir != "" {
  19. // TODO(mlaventure): how would we support that?
  20. return errdefs.Forbidden(errors.New("custom checkpointdir is not supported"))
  21. }
  22.  
  23. // if we encounter an error during start we need to ensure that any other
  24. // setup has been cleaned up properly
  25. defer func() {
  26. if err != nil {
  27. container.SetError(err)
  28. // if no one else has set it, make sure we don't leave it at zero
  29. if container.ExitCode() == 0 {
  30. container.SetExitCode(128)
  31. }
  32. if err := container.CheckpointTo(daemon.containersReplica); err != nil {
  33. logrus.Errorf("%s: failed saving state on start failure: %v", container.ID, err)
  34. }
  35. container.Reset(false)
  36.  
  37. daemon.Cleanup(container)
  38. // if containers AutoRemove flag is set, remove it after clean up
  39. if container.HostConfig.AutoRemove {
  40. container.Unlock()
  41. if err := daemon.ContainerRm(container.ID, &types.ContainerRmConfig{ForceRemove: true, RemoveVolume: true}); err != nil {
  42. logrus.Errorf("can't remove container %s: %v", container.ID, err)
  43. }
  44. container.Lock()
  45. }
  46. }
  47. }()
  48.  
  49. if err := daemon.conditionalMountOnStart(container); err != nil {
  50. return err
  51. }
  52.  
  53. if err := daemon.initializeNetworking(container); err != nil {
  54. return err
  55. }
  56.  
  57. spec, err := daemon.createSpec(container)
  58. if err != nil {
  59. return errdefs.System(err)
  60. }
  61.  
  62. if resetRestartManager {
  63. container.ResetRestartManager(true)
  64. container.HasBeenManuallyStopped = false
  65. }
  66.  
  67. if err := daemon.saveAppArmorConfig(container); err != nil {
  68. return err
  69. }
  70.  
  71. if checkpoint != "" {
  72. checkpointDir, err = getCheckpointDir(checkpointDir, checkpoint, container.Name, container.ID, container.CheckpointDir(), false)
  73. if err != nil {
  74. return err
  75. }
  76. }
  77.  
  78. shim, createOptions, err := daemon.getLibcontainerdCreateOptions(container)
  79. if err != nil {
  80. return err
  81. }
  82.  
  83. ctx := context.TODO()
  84.  
  85. err = daemon.containerd.Create(ctx, container.ID, spec, shim, createOptions)
  86. if err != nil {
  87. if errdefs.IsConflict(err) {
  88. logrus.WithError(err).WithField("container", container.ID).Error("Container not cleaned up from containerd from previous run")
  89. // best effort to clean up old container object
  90. daemon.containerd.DeleteTask(ctx, container.ID)
  91. if err := daemon.containerd.Delete(ctx, container.ID); err != nil && !errdefs.IsNotFound(err) {
  92. logrus.WithError(err).WithField("container", container.ID).Error("Error cleaning up stale containerd container object")
  93. }
  94. err = daemon.containerd.Create(ctx, container.ID, spec, shim, createOptions)
  95. }
  96. if err != nil {
  97. return translateContainerdStartErr(container.Path, container.SetExitCode, err)
  98. }
  99. }
  100.  
  101. // TODO(mlaventure): we need to specify checkpoint options here
  102. pid, err := daemon.containerd.Start(context.Background(), container.ID, checkpointDir,
  103. container.StreamConfig.Stdin() != nil || container.Config.Tty,
  104. container.InitializeStdio)
  105. if err != nil {
  106. if err := daemon.containerd.Delete(context.Background(), container.ID); err != nil {
  107. logrus.WithError(err).WithField("container", container.ID).
  108. Error("failed to delete failed start container")
  109. }
  110. return translateContainerdStartErr(container.Path, container.SetExitCode, err)
  111. }
  112.  
  113. container.HasBeenManuallyRestarted = false
  114. container.SetRunning(pid, true)
  115. container.HasBeenStartedBefore = true
  116. daemon.setStateCounter(container)
  117.  
  118. daemon.initHealthMonitor(container)
  119.  
  120. if err := container.CheckpointTo(daemon.containersReplica); err != nil {
  121. logrus.WithError(err).WithField("container", container.ID).
  122. Errorf("failed to store container")
  123. }
  124.  
  125. daemon.LogContainerEvent(container, "start")
  126. containerActions.WithValues("start").UpdateSince(start)
  127.  
  128. return nil
  129. }

3.2 PreStop

  源码分析见《详解Kubernetes Pod优雅退出》这篇博文,本文不再赘余。

4、postStart 异步执行

  在第三章节讲述开始容器方法时,我们已经很清晰知道,开始容器方法先运行容器,再运行运行容器 postStart 钩子函数

  需要注意的事,docker start 命令在启动容器时,并不会等待 ENTRYPOINT 命令执行完毕后才返回。它只会等待容器的主进程启动并确认其正常运行,然后立即返回。换句话说,docker start 命令返回时,ENTRYPOINT 命令已经开始执行,但不一定已经执行完毕。

  所以PostStart的执行相对于容器主进程的执行是异步的,它会在容器start后立刻触发,并不能保证PostStart钩子在容器ENTRYPOINT之前运行。

4.1 关于postStart异步执行测试

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4. name: test-post-start
  5. spec:
  6. containers:
  7. - name: test-post-start-container
  8. image: busybox
  9. command: ["/bin/sh", "-c", "sleep 5 && echo $(date) 'written by entrypoint' >> log.log && sleep 600"]
  10. lifecycle:
  11. postStart:
  12. exec:
  13. command: ["/bin/sh", "-c", "sleep 10 && echo $(date) 'written by post start' >> log.log"]

创建上面的pod,通过进入pod,查看log.log打印日志,证明:

  • PostStart是否会挡住主进程的启动
  • PostStart是否是异步执行

如果 PostStart 会阻挡 ENTRYPOINT 的启动,则日志文件内容应该是:

  1. (时间点 Twritten by post start
  2. (时间点 T + 10 秒)written by entrypoint

否则内容应该是:

  1. (时间点 Twritten by entrypoint
  2. (时间点 T + 5 秒)written by post start
  3. ```log

实验结果:

  1. / # cat log.log
  2. Thu Jun 4 06:14:50 UTC 2020 written by entrypoint
  3. Thu Jun 4 06:14:55 UTC 2020 written by post start

修改YAML文件:

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4. name: test-post-start
  5. spec:
  6. containers:
  7. - name: test-post-start-container
  8. image: busybox
  9. command: ["/bin/sh", "-c", "sleep 15 && echo $(date) 'written by entrypoint' >> log.log && sleep 600"]
  10. lifecycle:
  11. postStart:
  12. exec:
  13. command: ["/bin/sh", "-c", "sleep 10 && echo $(date) 'written by post start' >> log.log"]

如果 PostStart 不是异步执行,则日志文件内容应该是:

  1. (时间点 Twritten by entrypoint
  2. (时间点 T + 5 秒)written by post start
  3. ```log
  4.  
  5. 否则内容应该是:
  6. ```log
  7. (时间点 Twritten by post start
  8. (时间点 T + 5 秒)written by entrypoint

实验结果:

  1. [root@master k8s]# kubectl exec -it test-post-start sh
  2. / # cat log.log
  3. Thu Jun 4 06:17:54 UTC 2020 written by post start
  4. Thu Jun 4 06:17:59 UTC 2020 written by entrypoint
  5. / #

4.2 结论

  • PostStart不会挡住主进程的启动
  • PostStart是异步执行

5、使用场景

5.1 postStart 

  • 数据库连接:在容器启动后,可以使用 postStart 钩子来建立数据库连接,并确保应用程序在启动时可以正常访问数据库;
  • 文件下载:容器启动后,可以使用 postStart 钩子从外部下载文件,并将其放置在容器内部以供应用程序使用;
  • 启动后台任务:在容器启动后,可以使用 postStart 钩子来启动或触发后台任务,例如数据同步、日志收集等。

5.2 preStop

  • 优雅终止:在容器终止之前,可以使用 preStop 钩子发送信号或通知给应用程序,让应用程序优雅地处理未完成的请求或任务,并进行清理操作;
  • 数据持久化:在容器终止之前,可以使用 preStop 钩子将容器中的数据持久化到外部存储,以确保数据不会丢失;
  • 日志上传:在容器终止之前,可以使用 preStop 钩子将容器中的日志上传到中央日志系统,以便进一步分析和存档。

6、总结

  容器生命周期钩子(PostStart和PreStop)提供在容器启动后和停止前执行自定义操作的能力,适用于数据库连接、文件下载、优雅终止等场景。postStart平时用的场景不是很多,重点关注preStop。

参考: https://blog.51cto.com/happywzy/2855934

原文链接:https://www.cnblogs.com/zhangmingcheng/p/18264706

 友情链接:直通硅谷  点职佳  北美留学生论坛

本站QQ群:前端 618073944 | Java 606181507 | Python 626812652 | C/C++ 612253063 | 微信 634508462 | 苹果 692586424 | C#/.net 182808419 | PHP 305140648 | 运维 608723728

W3xue 的所有内容仅供测试,对任何法律问题及风险不承担任何责任。通过使用本站内容随之而来的风险与本站无关。
关于我们  |  意见建议  |  捐助我们  |  报错有奖  |  广告合作、友情链接(目前9元/月)请联系QQ:27243702 沸活量
皖ICP备17017327号-2 皖公网安备34020702000426号