scp will quietly skip directories it can't read

The short version

If you run scp -r over a tree that contains a directory you can’t read (or can’t search — the execute bit on a directory), scp will copy everything it can reach, print a permission denied line somewhere in the noise, and move on. The transfer looks like it finished. The missing subtree is just… not there.

I lost a chunk of a backup this way. Nothing crashed, nothing turned red. I only noticed because a directory I knew should be ~12 GB showed up as a few hundred MB on the other side.

Why it happens

Two permission bits on a directory matter here:

• read (r) — lets you list the names inside the directory.
• execute (x) — lets you enter the directory and stat/open the things inside it.

When scp -r recurses, it has to list each directory to know what to descend into. Hit one where you lack r (or x), and scp can’t enumerate its contents. It can’t copy what it can’t see, so it reports the error for that directory and carries on with the rest of the walk.

The catch is that this is treated as a per-entry failure, not a stop-the-world error. On a big tree the message scrolls past in seconds, and depending on your scp version the overall exit status doesn’t always make the partial failure obvious. “It printed a thousand filenames and came back to the prompt” reads as success when you’re not looking closely.

This bites hardest when:

• you’re copying someone else’s home directory, or a system path, as a non-root user;
• you sudo the local side of the copy but the remote side runs as your unprivileged user (or vice-versa);
• a single nested .cache, lost+found, or per-user directory in the middle of an otherwise-readable tree is locked down.

How to not get burned

Prefer rsync for anything that matters. It prints unreadable paths as clear rsync: ... failed: Permission denied errors and exits with a non-zero, documented status (24 = “partial transfer due to vanished/ inaccessible files”). You can also ask it for a summary:

rsync -a --info=stats2 src/ host:dest/
echo "exit: $?"   # anything non-zero = look closer

If you must use scp, check the exit code and don’t trust your eyes:

scp -r src host:dest || echo "scp reported errors — verify the tree"

Verify the result instead of assuming. A cheap sanity check is to compare file counts and sizes on both ends:

find src       -type f | wc -l
ssh host 'find dest -type f | wc -l'

Or sidestep permissions entirely by reading the tree as the user who can read it, and streaming a tarball:

sudo tar -C / -cf - path/to/tree | ssh host 'tar -C /dest -xf -'

tar will still complain about anything it can’t read, but at least the privileged side is doing the reading.

The takeaway

scp exiting cleanly means “scp finished walking”, not “everything got copied”. For backups, migrations, or anything you won’t get a second shot at, use a tool that treats an unreadable directory as a loud failure — or verify the destination yourself. A copy you didn’t check isn’t a copy you can count on.