Solved How to zfs send and receive between two servers on the same LAN without ssh?

S

sdf

Hello everyone,

How to zfs send and receive between two servers on the same LAN without ssh?

Non-encrypted transmission mode.

Thanks.
 
All it takes is a program which can process an incoming stream of data and send that to a remote host. An ftp client might be able to (I never tried this though). Still, sounds like a waste of effort to me.

What you could do is store the data stream into a file and then transfer that file using other methods, for example FTP. So you'd use zfs send to generate and store the data, then you could use something as rsync to get it stored onto a remote site. Maybe rsync can also utilize stdin but I'm not sure from mind.

This is roughly how my backup plan works. All my backup images are provided using HTTP (WAN only obviously), so when things go totally wrong then all I'd have to do is generate a HTTP stream (fetch can do that) and feed that into zfs receive. So I can restore data without having to depend on having to start any specific processes on my remote backup host.
 
You can do it with nc(1). Something like:

zfs send ... | nc dest_host 4242 on the sending side and nc -l 4242 | zfs recv ... on receiving. Start the receive side first since it will sit and block and wait for a connection.

If you are using a bash(1) shell for your scripting, you can even use zfs send ... > /dev/tcp/dest_host/4242 for the send. The listen side you need something like nc to create the listening port.

Any firewall will obviously need to be configured to permit the connection. For private networks (where you truly feel you have no security concerns) and especially on low-powered systems which may max out on ssh/CPU, this works well.

Also look at misc/mbuffer as another option; it can listen to / connect to network ports; I also find it can speed up the overall transfer with a reasonably-sized buffer (10s worth of transfer or more if you have the RAM available) especially on the receive end.
 
  • Thanks
Reactions: sdf
Eric A. Borisch said:
No, but you do need to install bash from ports.
Click to expand...
That doesn't seem correct. Bash is merely a shell which mainly relies on the OS to provide features such as these, see also bash(1):

Code: 
       Bash handles several filenames specially when they are used in redirec-
       tions, as described in the following table.  If the operating system on
       which bash is running provides these special files, bash will use them;
       otherwise  it  will emulate them internally with the behavior described
       below.

              /dev/fd/fd
                     If fd is a valid integer, file descriptor  fd  is  dupli-
                     cated.
              /dev/stdin
                     File descriptor 0 is duplicated.
              /dev/stdout
                     File descriptor 1 is duplicated.
              /dev/stderr
                     File descriptor 2 is duplicated.
              /dev/tcp/host/port
                     If host is a valid hostname or Internet address, and port
                     is an integer port number or service name, bash  attempts
                     to open the corresponding TCP socket.
I haven't extensively tested this, but when I run bash then I don't have /dev/tcp available, making me convinced that this is a Linux only feature.
 
Odd; it works for me. Did you try to browse into it, or to redirect output into a /dev/tcp/<hostname>/<port> ? Only the latter will work.

$ echo foo > /dev/tcp/remote-host/9999 on sending system.

and

$ nc -l 9999
foo
 on receiving system.

And for reference:
$pkg info bash
bash-4.4.23
Name : bash
Version : 4.4.23
Installed on : Wed Jun 6 22:53:15 2018 CDT
Origin : shells/bash
Architecture : FreeBSD:11:amd64
Prefix : /usr/local
Categories : shells
Licenses : GPLv3+
Maintainer : ehaupt@FreeBSD.org
WWW : https://www.gnu.org/software/bash/
Comment : GNU Project's Bourne Again SHell
Options :
COLONBREAKSWORDS: on
DOCS : on
FDESCFS : off
HELP : on
NLS : on
STATIC : off
SYSBASHRC : off
SYSLOG : off
Shared Libs required:
libintl.so.8
Annotations :
FreeBSD_version: 1101001
cpe : cpe:2.3:a:gnu:bash:4.4.23:::::freebsd11:x64
repo_type : binary
repository : Synth
Flat size : 7.75MiB
Description :
This is GNU Bash. Bash is the GNU Project's Bourne Again SHell,
a complete implementation of the POSIX.2 shell spec, but also
with interactive command line editing, job control on architectures
that support it, csh-like features such as history substitution and
brace expansion, and a slew of other features.

WWW: https://www.gnu.org/software/bash/


I believe those are also the default settings, but I didn't check.

You were in the bash shell when you tried, right?
 
And if the listening side isn't running when you initiate the send, you should see this:

$ echo foo > /dev/tcp/[I]remote[/I]/[I]port[/I]
-bash: connect: Connection refused
-bash: /dev/tcp/[I]remote[/I]/[I]port[/I]: Connection refused
 
sysutils/bbcp works for this. It uses SSH to authenticate the connection, but transfers the data in the clear, using multiple streams. It can easily saturate a gigabit link between ZFS servers (we don't have anything faster than that).

Code: 
# bbcp -4 c -N io "zfs send -I poolname/dataset@snapname1 poolname/dataset@snapname2" remoteuser@remoteserver:"zfs recv -d poolname"

That will login to remoteserver as remoteuser, start a listening bbcp process, then pipe the output of the zfs send through the bbcp transport, and pipe it out to the zfs recv on the other end.
 
You must log in or register to reply here.
Back
Top