Re: [gentoo-user] Docker installation issues

[Andreas Fink]

On Sat, 10 Dec 2022 12:30:40 -0500
Mansour Al Akeel <mansour.alakeel@×××××.com> wrote:

> I am using Openrc
>
> This was my initial /etc/conf.d/docker
> DOCKER_OPTS="--storage-driver overlay2 --data-root /srv/var/lib/docker"
>
> when I try:
>
> $ docker pull hello-world
>
> Error response from daemon: Get "https://registry-1.docker.io/v2/":
> net/http: request canceled while waiting for connection (Client.Timeout
> exceeded while awaiting headers)
>
> Trying to troubleshoot:
>
> localhost /home/mansour # tail -n 20 /var/log/docker.log
> time="2022-12-10T12:17:03.473550705-05:00" level=info msg="scheme \"unix\"
> not registered, fallback to default scheme" module=grpc
> time="2022-12-10T12:17:03.473566413-05:00" level=info
> msg="ccResolverWrapper: sending update to cc:
> {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}"
> module=grpc
> time="2022-12-10T12:17:03.473573787-05:00" level=info msg="ClientConn
> switching balancer to \"pick_first\"" module=grpc
> time="2022-12-10T12:17:03.474530993-05:00" level=info msg="parsed scheme:
> \"unix\"" module=grpc
> time="2022-12-10T12:17:03.474545549-05:00" level=info msg="scheme \"unix\"
> not registered, fallback to default scheme" module=grpc
> time="2022-12-10T12:17:03.474563752-05:00" level=info
> msg="ccResolverWrapper: sending update to cc:
> {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}"
> module=grpc
> time="2022-12-10T12:17:03.474571186-05:00" level=info msg="ClientConn
> switching balancer to \"pick_first\"" module=grpc
> time="2022-12-10T12:17:03.478908716-05:00" level=warning msg="Your kernel
> does not support cgroup blkio weight"
> time="2022-12-10T12:17:03.478927115-05:00" level=warning msg="Your kernel
> does not support cgroup blkio weight_device"
> time="2022-12-10T12:17:03.479037897-05:00" level=info msg="Loading
> containers: start."
> time="2022-12-10T12:17:03.495743563-05:00" level=info msg="failed to read
> ipv6 net.ipv6.conf.<bridge>.accept_ra" bridge=docker0
> syspath=/proc/sys/net/ipv6/conf/docker0/accept_ra
> time="2022-12-10T12:17:03.518761943-05:00" level=info msg="Default bridge
> (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip
> can be used to set a preferred IP address"
> time="2022-12-10T12:17:03.518886881-05:00" level=info msg="failed to read
> ipv6 net.ipv6.conf.<bridge>.accept_ra" bridge=docker0
> syspath=/proc/sys/net/ipv6/conf/docker0/accept_ra
> time="2022-12-10T12:17:03.534616741-05:00" level=info msg="Loading
> containers: done."
> time="2022-12-10T12:17:03.541080189-05:00" level=info msg="Docker daemon"
> commit=a89b84221c graphdriver(s)=overlay2 version=20.10.17
> time="2022-12-10T12:17:03.541122352-05:00" level=info msg="Daemon has
> completed initialization"
> time="2022-12-10T12:17:03.549888103-05:00" level=info msg="API listen on
> /var/run/docker.sock"
> time="2022-12-10T12:17:27.025622231-05:00" level=warning msg="Error getting
> v2 registry: Get \"https://registry-1.docker.io/v2/\": net/http: request
> canceled while waiting for connection (Client.Timeout exceeded while
> awaiting headers)"
> time="2022-12-10T12:17:27.025667054-05:00" level=info msg="Attempting next
> endpoint for pull after error: Get \"https://registry-1.docker.io/v2/\":
> net/http: request canceled while waiting for connection (Client.Timeout
> exceeded while awaiting headers)"
> time="2022-12-10T12:17:27.026851821-05:00" level=error msg="Handler for
> POST /v1.41/images/create returned error: Get \"
> https://registry-1.docker.io/v2/\": net/http: request canceled while
> waiting for connection (Client.Timeout exceeded while awaiting headers)"
>
> time="2022-12-10T12:15:42.036053086-05:00" level=info msg="loading plugin
> \"io.containerd.internal.v1.tracing\"..." type=io.containerd.internal.v1
> time="2022-12-10T12:15:42.036068412-05:00" level=error msg="failed to
> initialize a tracing processor \"otlp\"" error="no OpenTelemetry endpoint:
> skip plugin"
> time="2022-12-10T12:15:42.036100189-05:00" level=info msg="loading plugin
> \"io.containerd.grpc.v1.cri\"..." type=io.containerd.grpc.v1
> time="2022-12-10T12:15:42.036390695-05:00" level=info msg="Start cri plugin
> with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs
> DefaultRuntimeName:runc DefaultRuntime:{Type: Path: Engine:
> PodAnnotations:[] ContainerAnnotations:[] Root: Options:map[]
> PrivilegedWithoutHostDevices:false BaseRuntimeSpec: NetworkPluginConfDir:
> NetworkPluginMaxConfNum:0} UntrustedWorkloadRuntime:{Type: Path: Engine:
> PodAnnotations:[] ContainerAnnotations:[] Root: Options:map[]
> PrivilegedWithoutHostDevices:false BaseRuntimeSpec: NetworkPluginConfDir:
> NetworkPluginMaxConfNum:0} Runtimes:map[runc:{Type:io.containerd.runc.v2
> Path: Engine: PodAnnotations:[] ContainerAnnotations:[] Root:
> Options:map[BinaryName: CriuImagePath: CriuPath: CriuWorkPath: IoGid:0
> IoUid:0 NoNewKeyring:false NoPivotRoot:false Root: ShimCgroup:
> SystemdCgroup:false] PrivilegedWithoutHostDevices:false BaseRuntimeSpec:
> NetworkPluginConfDir: NetworkPluginMaxConfNum:0}] NoPivot:false
> DisableSnapshotAnnotations:true DiscardUnpackedLayers:false
> IgnoreRdtNotEnabledErrors:false}
> CniConfig:{NetworkPluginBinDir:/opt/cni/bin
> NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1
> NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath:
> Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]}
> ImageDecryption:{KeyModel:node} DisableTCPService:true
> StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s
> EnableSelinux:false SelinuxCategoryRange:1024 SandboxImage:
> k8s.gcr.io/pause:3.6 StatsCollectPeriod:10 SystemdCgroup:false
> EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:}
> MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false
> RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false
> UnsetSeccompProfile: TolerateMissingHugetlbController:true
> DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false
> IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false
> EnableUnprivilegedPorts:false EnableUnprivilegedICMP:false}
> ContainerdRootDir:/var/lib/containerd
> ContainerdEndpoint:/run/containerd/containerd.sock
> RootDir:/var/lib/containerd/io.containerd.grpc.v1.cri
> StateDir:/run/containerd/io.containerd.grpc.v1.cri}"
> time="2022-12-10T12:15:42.036530498-05:00" level=info msg="Connect
> containerd service"
> time="2022-12-10T12:15:42.036627681-05:00" level=info msg="Get image
> filesystem path
> \"/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs\""
> time="2022-12-10T12:15:42.036896077-05:00" level=error msg="failed to load
> cni during init, please check CRI plugin status before setting up network
> for pods" error="cni config load failed: no network config found in
> /etc/cni/net.d: cni plugin not initialized: failed to load cni config"
> time="2022-12-10T12:15:42.036986623-05:00" level=info msg="Start
> subscribing containerd event"
> time="2022-12-10T12:15:42.037028034-05:00" level=info msg="Start recovering
> state"
> time="2022-12-10T12:15:42.037035874-05:00" level=info msg=serving...
> address=/run/containerd/containerd.sock.ttrpc
> time="2022-12-10T12:15:42.037068181-05:00" level=info msg=serving...
> address=/run/containerd/containerd.sock
> time="2022-12-10T12:15:42.037085346-05:00" level=info msg="containerd
> successfully booted in 0.015889s"
> time="2022-12-10T12:15:42.037072303-05:00" level=info msg="Start event
> monitor"
> time="2022-12-10T12:15:42.037107987-05:00" level=info msg="Start snapshots
> syncer"
> time="2022-12-10T12:15:42.037119126-05:00" level=info msg="Start cni
> network conf syncer for default"
> time="2022-12-10T12:15:42.037127266-05:00" level=info msg="Start streaming
> server"
>
>
> Looking at resolv.conf
>
> localhost /home/mansour # cat /etc/resolv.conf
> # Generated by dhcpcd from wlp3s0.dhcp
> # /etc/resolv.conf.head can replace this line
> domain Home
> nameserver 192.168.1.1
> # /etc/resolv.conf.tail can replace this line
>
> So, I added the following to my /etc/dhcpcd.conf
> static domain_name_servers=8.8.8.8 8.8.4.4 192.168.1.1
>
> And restarted dhcpcd:
>
> localhost /home/mansour # cat /etc/resolv.conf
> # Generated by dhcpcd from wlp3s0.dhcp
> # /etc/resolv.conf.head can replace this line
> domain Home
> nameserver 8.8.8.8
> nameserver 8.8.4.4
> nameserver 192.168.1.1
> # /etc/resolv.conf.tail can replace this line
>
>
> docker.log still shows the same error:
> time="2022-12-10T12:28:16.519126445-05:00" level=warning msg="Error getting
> v2 registry: Get \"https://registry-1.docker.io/v2/\": net/http: request
> canceled while waiting for connection (Client.Timeout exceeded while
> awaiting headers)"
> time="2022-12-10T12:28:16.519173944-05:00" level=info msg="Attempting next
> endpoint for pull after error: Get \"https://registry-1.docker.io/v2/\":
> net/http: request canceled while waiting for connection (Client.Timeout
> exceeded while awaiting headers)"
> time="2022-12-10T12:28:16.520489790-05:00" level=error msg="Handler for
> POST /v1.41/images/create returned error: Get \"
> https://registry-1.docker.io/v2/\": net/http: request canceled while
> waiting for connection (Client.Timeout exceeded while awaiting headers)"
>
> I am not able to login with "docker login"
>
> Where do I look further for additional troubleshooting hints? Google didn't
> help a lot.
>
> Thank you.


Hello,
my first step would be checking if there is some inherent network
problem reaching registry-1.docker.io/v2, e.g. via
curl -v https://registry-1.docker.io/v2/

If that runs successfully, you could try if there is an MTU problem. I
remember that I had to set MTU to 1450 in some virtualized environment,
otherwise docker would not run correctly, i.e. I  had in
/etc/conf.d/docer
DOCKER_OPTS="--mtu=1450"

Hope that gives you an idea what to look for.

If both does not help, there is wireshark, to watch your network, very
low-level, but might give some hints.

Cheers
Andreas