Table of Contents

Proxmox VE Troubleshooting Guide

Table of Contents

Repository Issues

Error: The repository 'https://download.docker.com/linux/ubuntu bookworm Release' does not have a Release file

Problem: When attempting to install Docker on Proxmox using the official Docker package manager steps, you may encounter this error.

Hit:5 http://download.proxmox.com/debian/pve bookworm InRelease
Hit:6 http://download.proxmox.com/debian/ceph-quincy bookworm InRelease
Ign:7 https://download.docker.com/linux/ubuntu bookworm InRelease
Err:8 https://download.docker.com/linux/ubuntu bookworm Release
  404  Not Found [IP: 54.230.202.11 443]
Reading package lists... Done
E: The repository 'https://download.docker.com/linux/ubuntu bookworm Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Root Cause: Proxmox is based on Debian, not Ubuntu. The Docker repository for Ubuntu Bookworm does not exist, but there is one for Debian Bookworm. The default instructions from the Docker website may add an Ubuntu source, causing this error.

Solution:

  1. Open the Docker repository source file for editing:

    sudo nano /etc/apt/sources.list.d/docker.list

  2. Change the entry from ubuntu to debian:

    # From:
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu bookworm stable
# To:
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian bookworm stable

  3. Update package lists:

    sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io

Error: "repository 'https://enterprise.proxmox.com/debian/pve' doesn't have a Release file"

Problem: Subscription repository errors when trying to update packages.

Solution:

  1. Disable enterprise repository:

    sudo sed -i '/^deb/s/^/# /' /etc/apt/sources.list.d/pve-enterprise.list

  2. Add no-subscription repository:

    echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" | sudo tee -a /etc/apt/sources.list

  3. Update package lists:

    sudo apt update

This should resolve the error and allow you to install Docker on Proxmox successfully.

Installation Problems

Boot Issues During Installation

Problem: System fails to boot from installation media or gets stuck during boot.

Solutions:

  1. UEFI/BIOS Settings:

    • Enable virtualization extensions (VT-x/AMD-V)
    • Disable Secure Boot temporarily
    • Set boot mode to UEFI (recommended) or Legacy

  2. Hardware Compatibility:

    # Check hardware virtualization support
grep -E '(vmx|svm)' /proc/cpuinfo

  3. Memory Issues:

    • Ensure minimum 2GB RAM available
    • Test memory with memtest86+ if installation fails randomly

Installation Hangs at "Starting PVE Manager"

Problem: Installation process hangs indefinitely.

Solution:

  1. Disable IPv6 during installation:

    • Add ipv6.disable=1 to kernel boot parameters
    • Press 'e' at GRUB menu and add to linux line

  2. Network configuration:

    • Use static IP instead of DHCP during installation
    • Ensure network cable is properly connected

Network Connectivity

No Network Access After Installation

Problem: Cannot access web interface or network connectivity is broken.

Diagnosis:

# Check network interface status
ip addr show
ip route show

# Test connectivity
ping 8.8.8.8
ping google.com

# Check network service
systemctl status networking

Solutions:

  1. Reconfigure network:

    # Edit network configuration
nano /etc/network/interfaces

# Example configuration:
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
        address 192.168.1.100/24
        gateway 192.168.1.1
        dns-nameservers 8.8.8.8 8.8.4.4

auto vmbr0
iface vmbr0 inet static
        address 192.168.1.100/24
        gateway 192.168.1.1
        bridge-ports ens18
        bridge-stp off
        bridge-fd 0

  2. Restart networking:

    systemctl restart networking

VLAN Issues

Problem: VLANs not working properly with VMs.

Solution:

# Enable VLAN aware on bridge
pvesh set /nodes/$(hostname)/network/vmbr0 --vlan-aware 1

# Restart networking
systemctl restart networking

Virtual Machine Issues

VM Won't Start - Hardware Virtualization

Problem: Error messages about hardware virtualization not being available.

Diagnosis:

# Check if hardware virtualization is enabled
grep -E '(vmx|svm)' /proc/cpuinfo

# Check if KVM modules are loaded
lsmod | grep kvm

Solutions:

  1. Enable in BIOS/UEFI:

    • Intel: Enable VT-x
    • AMD: Enable AMD-V
    • Reboot and check BIOS settings

  2. Load KVM modules:

    # For Intel
modprobe kvm-intel

# For AMD
modprobe kvm-amd

# Make permanent
echo "kvm-intel" >> /etc/modules  # or kvm-amd

VM Performance Issues

Problem: Virtual machines running slowly or experiencing high CPU wait times.

Diagnosis:

# Check VM resource usage
qm monitor <vmid>
info cpus
info memory

# Check host resources
top
iostat -x 1

Solutions:

  1. CPU Configuration:

    • Set CPU type to "host" for better performance
    • Enable CPU flags in VM configuration
    • Adjust CPU cores based on workload

  2. Memory Management:

    # Enable ballooning
qm set <vmid> --balloon 1

# Set appropriate memory limits
qm set <vmid> --memory 4096 --balloon 2048

  3. Storage Optimization:

    • Use virtio-scsi for disk controller
    • Enable write-back cache for better performance
    • Consider using SSD storage for VMs

VM Backup Failures

Problem: Backup operations fail or take too long.

Diagnosis:

# Check backup logs
tail -f /var/log/pve/tasks/active

# Check storage space
df -h
pvesm status

Solutions:

  1. Storage Issues:

    # Clean up old backups
find /var/lib/vz/dump -name "*.vma*" -mtime +30 -delete

# Check storage configuration
pvesm status

  2. Backup Configuration:

    • Enable snapshot mode for faster backups
    • Exclude unnecessary disks from backup
    • Schedule backups during low-usage hours

Container Problems

LXC Container Won't Start

Problem: Linux containers fail to start with various errors.

Diagnosis:

# Check container status
pct status <ctid>

# View container logs
pct list
journalctl -u pve-container@<ctid>

Solutions:

  1. Privilege Issues:

    # Convert to privileged container if needed
pct set <ctid> --unprivileged 0

  2. Resource Limits:

    # Check and adjust memory limits
pct set <ctid> --memory 1024 --swap 512

# Check disk space
pct set <ctid> --rootfs local-lvm:8

Container Network Issues

Problem: Containers cannot reach network or internet.

Solutions:

# Check container network configuration
pct config <ctid>

# Set static IP for container
pct set <ctid> --net0 name=eth0,bridge=vmbr0,ip=192.168.1.101/24,gw=192.168.1.1

# Restart container
pct restart <ctid>

Storage Issues

Storage Full or Performance Problems

Problem: Storage space exhausted or poor I/O performance.

Diagnosis:

# Check storage usage
df -h
pvesm status
zpool status  # If using ZFS

# Check I/O performance
iostat -x 1 5
iotop

Solutions:

  1. Clean up space:

    # Remove old backups
find /var/lib/vz/dump -name "*.vma*" -mtime +7 -delete

# Clean package cache
apt clean

# Remove unused kernels
apt autoremove

  2. ZFS specific:

    # Check ZFS pool health
zpool status
zpool list

# Scrub ZFS pool
zpool scrub rpool

Disk Errors and Failures

Problem: Disk errors or drive failures affecting storage.

Diagnosis:

# Check for disk errors
dmesg | grep -i error
smartctl -a /dev/sda

# Check filesystem
fsck /dev/sda1

Solutions:

# Replace failed drive in ZFS
zpool replace rpool /dev/old-disk /dev/new-disk

# Monitor disk health
smartctl -t long /dev/sda
smartctl -l selftest /dev/sda

Performance Problems

High CPU Usage

Problem: Host system experiencing high CPU utilization.

Diagnosis:

# Check processes
top
htop
ps aux --sort=-%cpu | head

# Check VM CPU usage
qm monitor <vmid>
info cpus

Solutions:

  1. CPU Scaling:

    # Check CPU governor
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

# Set performance governor
echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

  2. VM Configuration:

    • Adjust CPU limits for VMs
    • Enable CPU ballooning
    • Set appropriate CPU types

Memory Issues

Problem: High memory usage or out-of-memory conditions.

Diagnosis:

# Check memory usage
free -h
cat /proc/meminfo
ps aux --sort=-%mem | head

Solutions:

# Enable KSM (Kernel Samepage Merging)
systemctl enable ksmtuned
systemctl start ksmtuned

# Adjust VM memory allocation
qm set <vmid> --balloon 1
qm set <vmid> --memory 2048 --balloon 1024

Web Interface Issues

Cannot Access Web Interface

Problem: Proxmox web interface is not accessible.

Diagnosis:

# Check web services
systemctl status pveproxy
systemctl status pvedaemon

# Check if ports are listening
netstat -tulpn | grep :8006
ss -tulpn | grep :8006

# Check firewall
iptables -L

Solutions:

  1. Restart web services:

    systemctl restart pveproxy
systemctl restart pvedaemon

  2. Certificate issues:

    # Regenerate certificates
pvecm updatecerts
systemctl restart pveproxy

  3. Firewall configuration:

    # Allow web interface through firewall
iptables -A INPUT -p tcp --dport 8006 -j ACCEPT

Subscription Popup

Problem: Constant subscription popup messages.

Solution:

# Disable subscription popup
sed -Ezi.bak "s/(Ext.Msg.show\(\{\s+title: gettext\('No valid sub)/void\(\{ \/\/\1/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
systemctl restart pveproxy

Backup and Restore

Backup Failures

Problem: VM or container backups failing.

Common Issues and Solutions:

  1. Insufficient Storage:

    # Check backup storage
pvesm status
df -h /var/lib/vz/dump

  2. Snapshot Issues:

    # Check for stale snapshots
qm listsnapshot <vmid>

# Remove old snapshots
qm delsnapshot <vmid> <snapshot-name>

  3. Guest Agent Issues:

    # Install guest agent in VM
apt install qemu-guest-agent  # Debian/Ubuntu
systemctl enable qemu-guest-agent
systemctl start qemu-guest-agent

Clustering Issues

Node Communication Problems

Problem: Cluster nodes cannot communicate properly.

Diagnosis:

# Check cluster status
pvecm status
pvecm nodes

# Check corosync
systemctl status corosync
corosync-quorumtool -s

Solutions:

  1. Network connectivity:

    # Test connectivity between nodes
ping <node-ip>

# Check cluster network
pvecm mtunnel -migration_network <network>

  2. Time synchronization:

    # Ensure NTP is synchronized
systemctl status ntp
ntpq -p

Log File Locations

System Logs

  • /var/log/syslog - General system logs
  • /var/log/daemon.log - Daemon logs (including Proxmox)
  • /var/log/auth.log - Authentication logs
  • /var/log/kern.log - Kernel logs

Proxmox Specific Logs

  • /var/log/pveproxy/access.log - Web interface access logs
  • /var/log/pve/tasks/ - Task execution logs
  • /var/log/libvirt/qemu/ - VM logs
  • /var/log/pve-firewall.log - Firewall logs

Checking Logs

# Real-time log monitoring
tail -f /var/log/syslog
tail -f /var/log/pve/tasks/active

# Search for specific errors
grep -i error /var/log/syslog
grep -i failed /var/log/daemon.log

# Journal logs
journalctl -f
journalctl -u pveproxy
journalctl -u qemu-server@<vmid>

Diagnostic Commands

System Information

# Hardware information
lscpu
lshw -short
dmidecode -t memory

# PCI devices
lspci
lspci | grep -i vga

# USB devices
lsusb

# Block devices
lsblk
blkid

Network Diagnostics

# Network configuration
ip addr show
ip route show
cat /etc/network/interfaces

# Network testing
ping -c 4 google.com
traceroute google.com
nslookup google.com

# Port testing
telnet <host> <port>
nc -zv <host> <port>

Storage Diagnostics

# Disk usage
df -h
du -sh /*

# Storage performance
iostat -x 1 5
iotop -o

# SMART disk health
smartctl -a /dev/sda
smartctl -t short /dev/sda

Proxmox Specific Commands

# VM management
qm list
qm status <vmid>
qm config <vmid>
qm monitor <vmid>

# Container management
pct list
pct status <ctid>
pct config <ctid>

# Storage management
pvesm status
pvesm list <storage>

# Cluster management (if clustered)
pvecm status
pvecm nodes

Quick Reference

Emergency Recovery

  1. Boot into rescue mode from installation media

  2. Mount root filesystem:

    mount /dev/sda3 /mnt  # Adjust device as needed
chroot /mnt

  3. Fix configuration and reboot

Performance Tuning Checklist

  • [ ] Enable hardware virtualization in BIOS
  • [ ] Configure appropriate CPU types for VMs
  • [ ] Enable memory ballooning where appropriate
  • [ ] Use virtio drivers for network and storage
  • [ ] Configure appropriate storage backend
  • [ ] Monitor resource usage regularly
  • [ ] Keep system updated
  • [ ] Configure proper backup strategies

Note: Always backup important data before making significant configuration changes. Test solutions in a non-production environment first when possible.

Edit this page