Solved zfs send | ssh zfs receive - how to stop zfs send from sending output until I have authenticated?

tOsYZYny

Active Member

Reaction score: 15
Messages: 213

I am using zfs send -v <SNAPSHOT> | ssh <TARGET> "zfs receive <TARGET_VOLUME>" to synchronize a volume. Before I authenticate, zfs send is already sending output to stdout saying:

This message repeats until I authenticate ...
00:30:57 63.2K <SNAPSHOT>
00:30:58 63.2K <SNAPSHOT>

Whenever I am authenticated, the data sent actually increases. Is there anyway for the me to stop output until I actually send data (when I'm authenticated)?
 

PMc

Daemon

Reaction score: 685
Messages: 1,381

I suppose this is from invoking zfs send -v, according to the manpage:
Print verbose information about the stream package generated.
This information includes a per-second report of how much
data has been sent.

henever I am authenticated, the data sent actually increases. Is there anyway for the me to stop output until I actually send data (when I'm authenticated)?

No.
zfs send is invoked as the first element of the pipe, so it must start it's work (and part of that is, according to -v, report the status every second). Then it fills the pipe's buffers (apparently near 64k) and waits until that is consumed by the next command in the pipe. And that would be ssh, and that has to wait for authentication before it can invoke zfs recv.
Everything's working as it should.
 
OP
T

tOsYZYny

Active Member

Reaction score: 15
Messages: 213

Ah, yes, you're right - okay, I skipped over that part. I think my only option then is to do away with the -v (verbose) flag and if it is hung, a timeout on that command will do the trick.
 

Eric A. Borisch

Aspiring Daemon

Reaction score: 362
Messages: 590

You could also set up public key authentication for your user between the two hosts. With this approach, there won’t be a password prompt for the SSH connection.

You can even add a password to (encrypt) the key itself (if you’re concerned about the state of the security on the originating system, or just want another layer of security) and provide it before invoking ssh using ssh-agent and ssh-add.
 

Eric A. Borisch

Aspiring Daemon

Reaction score: 362
Messages: 590

That said, I find sticking sysutils/pv in the pipeline is much nicer than the exceedingly verbose zfs send -v. If you dry-run the send first with -nP to see the total send size, you can even get a completion percentage/time remaining with zfs send [...] | pv -S <reported send size> | ssh [...].
 
OP
T

tOsYZYny

Active Member

Reaction score: 15
Messages: 213

Thanks, yes, with public key authentication, I don't experience that problem.

Okay, I didn't know that, that might be a future enhancement to my script.
 
Top