transfer data between computers

 1. netcat

receiving machine:

iptables -A INPUT -p tcp --dport 1440 -s $remoteIP -i br0 -j ACCEPT

nc -l -p 1440 |  dd bs=16M of=destimage

sending machine

pv -r -t -b -e sourceimage.iso |nc virt3 1440 -i 1

(dd or cat can be used instead of pv)
(netcat option -i 1 (idle timer) is -q 1 at other implementation)

with compression: 

serv1# nc -l -p 1440 | pigz -d | dd bs=16M of=/path/destimage

serv2# dd if=srcimage bs=16M |pv|pigz -1|nc virt3 1440 -i 1

in one command:

(zstd je alternativa, mozna rychlejsi)

ssh -n -f dest.example.com "bash -c 'nc -l -p 1440 |pigz -d |dd bs=16M of=/mnt/1/destimage>/dev/null 2>&1 &'" ;
sleep 2 ; dd if=srcimage bs=16M |pv|pigz -1|nc virt3 1440 -i 1 

2. ssh: 

 cat /dev/xxx | zstd | ssh dest.example.com "zstd -d | dd of=/dev/xxx" (nezkoušeno)

Optimal setting for badblocks

See: how-can-i-choose-optimal-values-for- ... -badblocks parameters?

Summary:

There is a badblocks benchmark script available that should suit your purpose.

 

1. The default size is  of data is 4M x 512 bytes, i.e., 2GB ; not too much.

2. The script performs destructive test on first 2GB of the device.

3. The cases for count=1 or 2 are slow and can be skipped.

typical result for 4k block (TOSHIBA MG04ACA600E, sector size 512 logical, 4096 physical)

2:39.77s: badblocks -b 4096 -c 2048
2:39.97s: badblocks -b 4096 -c 4096
2:40.43s: badblocks -b 4096 -c 1024
2:41.14s: badblocks -b 4096 -c 8192
2:41.39s: badblocks -b 4096 -c 512
2:42.29s: badblocks -b 4096 -c 256
2:43.70s: badblocks -b 4096 -c 16384
2:43.76s: badblocks -b 4096 -c 128
2:45.03s: badblocks -b 4096 -c 32768
2:45.83s: badblocks -b 4096 -c 65536
2:48.80s: badblocks -b 4096 -c 131072
2:52.29s: badblocks -b 4096 -c 64
2:59.83s: badblocks -b 4096 -c 32
3:15.19s: badblocks -b 4096 -c 16
3:36.80s: badblocks -b 4096 -c 8
4:25.55s: badblocks -b 4096 -c 4
6:23.96s: badblocks -b 4096 -c 2
10:33.7s: badblocks -b 4096 -c 1 

The full script run (TOSHIBA MG04ACA600E, sector size 512 logical, 4096 physical)

 

RUN1:                                      RUN2:    
0:19.57s: badblocks -b 512 -c 4             0:20.02s: badblocks -b 1024 -c 1
0:19.58s: badblocks -b 512 -c 2             0:20.09s: badblocks -b 512 -c 2
0:20.00s: badblocks -b 2048 -c 1            0:20.41s: badblocks -b 512 -c 4
0:20.57s: badblocks -b 1024 -c 2            0:20.48s: badblocks -b 2048 -c 1
0:20.67s: badblocks -b 8192 -c 1024         0:20.70s: badblocks -b 32768 -c 128
0:20.69s: badblocks -b 2048 -c 2048         0:20.73s: badblocks -b 16384 -c 128
0:20.71s: badblocks -b 32768 -c 128         0:20.73s: badblocks -b 512 -c 8192
0:20.75s: badblocks -b 512 -c 32768         0:20.73s: badblocks -b 65536 -c 128
0:20.75s: badblocks -b 65536 -c 128         0:20.74s: badblocks -b 1024 -c 8192
0:20.77s: badblocks -b 16384 -c 512         0:20.74s: badblocks -b 16384 -c 512
0:20.77s: badblocks -b 8192 -c 512          0:20.76s: badblocks -b 2048 -c 2048
0:20.78s: badblocks -b 1024 -c 8192         0:20.76s: badblocks -b 32768 -c 256 
0:20.78s: badblocks -b 16384 -c 1024        0:20.76s: badblocks -b 4096 -c 2048
0:20.78s: badblocks -b 32768 -c 256         0:20.76s: badblocks -b 8192 -c 1024
0:20.78s: badblocks -b 32768 -c 64          0:20.77s: badblocks -b 2048 -c 4096
0:20.78s: badblocks -b 4096 -c 1024         0:20.78s: badblocks -b 1024 -c 16384
0:20.78s: badblocks -b 512 -c 8192          0:20.78s: badblocks -b 8192 -c 512
0:20.78s: badblocks -b 65536 -c 256         0:20.79s: badblocks -b 16384 -c 1024
0:20.79s: badblocks -b 512 -c 16384         0:20.79s: badblocks -b 512 -c 16384
0:20.80s: badblocks -b 4096 -c 2048         0:20.80s: badblocks -b 2048 -c 8192
0:20.82s: badblocks -b 1024 -c 2048         0:20.80s: badblocks -b 4096 -c 1024
0:20.82s: badblocks -b 2048 -c 512          0:20.80s: badblocks -b 65536 -c 32
0:20.82s: badblocks -b 4096 -c 512          0:20.81s: badblocks -b 1024 -c 2048
0:20.83s: badblocks -b 16384 -c 256         0:20.81s: badblocks -b 32768 -c 512
0:20.83s: badblocks -b 16384 -c 64          0:20.82s: badblocks -b 1024 -c 4096
0:20.83s: badblocks -b 65536 -c 32          0:20.82s: badblocks -b 65536 -c 64
0:20.84s: badblocks -b 4096 -c 4096         0:20.83s: badblocks -b 4096 -c 4096
0:20.85s: badblocks -b 32768 -c 512         0:20.83s: badblocks -b 8192 -c 2048
0:20.85s: badblocks -b 65536 -c 64          0:20.84s: badblocks -b 32768 -c 64
0:20.86s: badblocks -b 512 -c 4096          0:20.84s: badblocks -b 8192 -c 256
0:20.86s: badblocks -b 8192 -c 2048         0:20.85s: badblocks -b 16384 -c 256
0:20.87s: badblocks -b 4096 -c 256          0:20.85s: badblocks -b 4096 -c 512
0:20.88s: badblocks -b 2048 -c 4096         0:20.85s: badblocks -b 512 -c 2048
0:20.89s: badblocks -b 2048 -c 1024         0:20.85s: badblocks -b 512 -c 4096
0:20.90s: badblocks -b 1024 -c 16384        0:20.89s: badblocks -b 65536 -c 16
0:20.90s: badblocks -b 2048 -c 16384        0:20.90s: badblocks -b 512 -c 32768
0:20.90s: badblocks -b 2048 -c 8192         0:20.91s: badblocks -b 2048 -c 1024
0:20.91s: badblocks -b 16384 -c 2048        0:20.95s: badblocks -b 1024 -c 1024
0:20.92s: badblocks -b 16384 -c 128         0:20.95s: badblocks -b 2048 -c 16384
0:20.93s: badblocks -b 65536 -c 16          0:20.97s: badblocks -b 65536 -c 256
0:20.95s: badblocks -b 4096 -c 8192         0:20.98s: badblocks -b 4096 -c 256
0:20.96s: badblocks -b 512 -c 1024          0:21.03s: badblocks -b 2048 -c 512
0:20.98s: badblocks -b 1024 -c 4096         0:21.04s: badblocks -b 32768 -c 32
0:20.99s: badblocks -b 1024 -c 1024         0:21.05s: badblocks -b 16384 -c 2048
0:20.99s: badblocks -b 8192 -c 128          0:21.05s: badblocks -b 8192 -c 4096
0:21.00s: badblocks -b 512 -c 2048          0:21.08s: badblocks -b 16384 -c 32
0:21.03s: badblocks -b 32768 -c 32          0:21.09s: badblocks -b 32768 -c 1024
0:21.04s: badblocks -b 8192 -c 4096         0:21.12s: badblocks -b 4096 -c 8192
0:21.05s: badblocks -b 65536 -c 512         0:21.12s: badblocks -b 512 -c 1024
0:21.06s: badblocks -b 32768 -c 1024        0:21.13s: badblocks -b 32768 -c 16
0:21.09s: badblocks -b 4096 -c 128          0:21.14s: badblocks -b 8192 -c 128
0:21.09s: badblocks -b 65536 -c 8           0:21.15s: badblocks -b 65536 -c 512
0:21.12s: badblocks -b 8192 -c 64           0:21.18s: badblocks -b 4096 -c 128
0:21.16s: badblocks -b 1024 -c 512          0:21.18s: badblocks -b 8192 -c 64
0:21.19s: badblocks -b 8192 -c 256          0:21.22s: badblocks -b 16384 -c 64
0:21.27s: badblocks -b 16384 -c 32          0:21.24s: badblocks -b 1024 -c 512
0:21.27s: badblocks -b 32768 -c 16          0:21.28s: badblocks -b 65536 -c 8
0:21.39s: badblocks -b 512 -c 65536         0:21.29s: badblocks -b 2048 -c 256
0:21.43s: badblocks -b 2048 -c 256          0:21.31s: badblocks -b 1024 -c 2
0:21.47s: badblocks -b 1024 -c 32768        0:21.46s: badblocks -b 1024 -c 32768
0:21.53s: badblocks -b 1024 -c 1            0:21.58s: badblocks -b 512 -c 65536
0:21.56s: badblocks -b 512 -c 512           0:21.64s: badblocks -b 512 -c 512
0:21.59s: badblocks -b 8192 -c 32           0:21.67s: badblocks -b 4096 -c 16384
0:21.64s: badblocks -b 16384 -c 4096        0:21.69s: badblocks -b 65536 -c 1024
0:21.66s: badblocks -b 8192 -c 8192         0:21.72s: badblocks -b 16384 -c 4096
0:21.67s: badblocks -b 65536 -c 1024        0:21.72s: badblocks -b 2048 -c 32768
0:21.69s: badblocks -b 32768 -c 2048        0:21.72s: badblocks -b 8192 -c 8192
0:21.77s: badblocks -b 2048 -c 32768        0:21.73s: badblocks -b 32768 -c 2048
0:21.78s: badblocks -b 4096 -c 16384        0:21.79s: badblocks -b 512 -c 131072
0:21.78s: badblocks -b 512 -c 131072        0:21.83s: badblocks -b 1024 -c 65536
0:21.88s: badblocks -b 1024 -c 65536        0:21.88s: badblocks -b 16384 -c 16
0:22.03s: badblocks -b 16384 -c 16          0:21.98s: badblocks -b 2048 -c 128
0:22.04s: badblocks -b 1024 -c 256          0:22.02s: badblocks -b 1024 -c 256
0:22.21s: badblocks -b 65536 -c 4           0:22.08s: badblocks -b 8192 -c 32
0:22.24s: badblocks -b 2048 -c 128          0:22.20s: badblocks -b 32768 -c 8
0:22.40s: badblocks -b 4096 -c 32768        0:22.21s: badblocks -b 4096 -c 64   
0:22.41s: badblocks -b 65536 -c 2048        0:22.30s: badblocks -b 4096 -c 32768
0:22.41s: badblocks -b 8192 -c 16384        0:22.41s: badblocks -b 16384 -c 8192
0:22.43s: badblocks -b 16384 -c 8192        0:22.42s: badblocks -b 2048 -c 65536
0:22.46s: badblocks -b 32768 -c 4096        0:22.45s: badblocks -b 32768 -c 4096
0:22.49s: badblocks -b 1024 -c 131072       0:22.46s: badblocks -b 65536 -c 2048
0:22.53s: badblocks -b 2048 -c 65536        0:22.46s: badblocks -b 8192 -c 16384
0:22.53s: badblocks -b 8192 -c 16           0:22.55s: badblocks -b 1024 -c 131072
0:22.72s: badblocks -b 32768 -c 8           0:22.63s: badblocks -b 65536 -c 4
0:22.75s: badblocks -b 4096 -c 64           0:22.85s: badblocks -b 4096 -c 32
0:22.86s: badblocks -b 512 -c 256           0:22.95s: badblocks -b 512 -c 256
0:23.22s: badblocks -b 65536 -c 2           0:23.02s: badblocks -b 32768 -c 4
0:23.26s: badblocks -b 4096 -c 32           0:23.04s: badblocks -b 1024 -c 128
0:23.27s: badblocks -b 1024 -c 128          0:23.16s: badblocks -b 2048 -c 64
0:23.32s: badblocks -b 16384 -c 8           0:23.31s: badblocks -b 65536 -c 2
0:23.32s: badblocks -b 32768 -c 4           0:23.41s: badblocks -b 8192 -c 16
0:23.48s: badblocks -b 2048 -c 64           0:23.54s: badblocks -b 16384 -c 8
0:23.75s: badblocks -b 2048 -c 131072       0:23.75s: badblocks -b 16384 -c 16384
0:23.78s: badblocks -b 16384 -c 16384       0:23.75s: badblocks -b 8192 -c 32768
0:23.79s: badblocks -b 32768 -c 8192        0:23.76s: badblocks -b 4096 -c 65536
0:23.79s: badblocks -b 4096 -c 65536        0:23.81s: badblocks -b 65536 -c 4096
0:23.80s: badblocks -b 8192 -c 32768        0:23.85s: badblocks -b 32768 -c 8192
0:23.84s: badblocks -b 65536 -c 4096        0:23.86s: badblocks -b 2048 -c 131072
0:25.08s: badblocks -b 65536 -c 1           0:25.13s: badblocks -b 2048 -c 32
0:25.16s: badblocks -b 4096 -c 16           0:25.23s: badblocks -b 8192 -c 8
0:25.25s: badblocks -b 32768 -c 2           0:25.31s: badblocks -b 32768 -c 2
0:25.32s: badblocks -b 8192 -c 8            0:25.45s: badblocks -b 512 -c 128
0:25.41s: badblocks -b 512 -c 128           0:25.51s: badblocks -b 4096 -c 16
0:25.52s: badblocks -b 16384 -c 4           0:25.55s: badblocks -b 65536 -c 1
0:25.72s: badblocks -b 2048 -c 32           0:25.56s: badblocks -b 1024 -c 64
0:26.12s: badblocks -b 1024 -c 64           0:26.15s: badblocks -b 16384 -c 4
0:26.55s: badblocks -b 4096 -c 131072       0:26.48s: badblocks -b 8192 -c 65536
0:26.56s: badblocks -b 16384 -c 32768       0:26.53s: badblocks -b 4096 -c 131072
0:26.56s: badblocks -b 32768 -c 16384       0:26.56s: badblocks -b 16384 -c 32768
0:26.56s: badblocks -b 65536 -c 8192        0:26.59s: badblocks -b 32768 -c 16384
0:26.57s: badblocks -b 8192 -c 65536        0:26.69s: badblocks -b 65536 -c 8192
0:28.07s: badblocks -b 512 -c 1             0:26.86s: badblocks -b 512 -c 1
0:30.16s: badblocks -b 8192 -c 4            0:29.74s: badblocks -b 8192 -c 4
0:30.18s: badblocks -b 32768 -c 1           0:29.79s: badblocks -b 4096 -c 8
0:30.31s: badblocks -b 4096 -c 8            0:29.93s: badblocks -b 1024 -c 32
0:30.35s: badblocks -b 512 -c 64            0:29.98s: badblocks -b 2048 -c 16
0:30.38s: badblocks -b 2048 -c 16           0:30.05s: badblocks -b 512 -c 64
0:30.44s: badblocks -b 1024 -c 32           0:30.35s: badblocks -b 32768 -c 1
0:30.84s: badblocks -b 16384 -c 2           0:30.92s: badblocks -b 16384 -c 2
0:32.06s: badblocks -b 16384 -c 65536       0:32.10s: badblocks -b 16384 -c 65536
0:32.09s: badblocks -b 8192 -c 131072       0:32.10s: badblocks -b 8192 -c 131072
0:32.15s: badblocks -b 65536 -c 16384       0:32.12s: badblocks -b 32768 -c 32768
0:32.19s: badblocks -b 32768 -c 32768       0:32.17s: badblocks -b 65536 -c 16384
0:39.88s: badblocks -b 1024 -c 16           0:39.48s: badblocks -b 16384 -c 1
0:40.33s: badblocks -b 4096 -c 4            0:39.88s: badblocks -b 2048 -c 8
0:40.40s: badblocks -b 16384 -c 1           0:39.99s: badblocks -b 512 -c 32
0:40.48s: badblocks -b 512 -c 32            0:40.00s: badblocks -b 4096 -c 4
0:40.56s: badblocks -b 8192 -c 2            0:40.12s: badblocks -b 1024 -c 16
0:41.01s: badblocks -b 2048 -c 8            0:40.28s: badblocks -b 8192 -c 2
0:58.43s: badblocks -b 8192 -c 1            0:58.04s: badblocks -b 4096 -c 2
0:58.75s: badblocks -b 512 -c 16            0:58.08s: badblocks -b 1024 -c 8
0:58.98s: badblocks -b 1024 -c 8            0:58.08s: badblocks -b 2048 -c 4
0:59.31s: badblocks -b 4096 -c 2            0:58.46s: badblocks -b 8192 -c 1
1:00.85s: badblocks -b 2048 -c 4            0:59.13s: badblocks -b 512 -c 16
1:38.71s: badblocks -b 2048 -c 2            1:38.28s: badblocks -b 2048 -c 2
1:38.87s: badblocks -b 1024 -c 4            1:38.43s: badblocks -b 512 -c 8
1:39.77s: badblocks -b 4096 -c 1            1:39.55s: badblocks -b 1024 -c 4
1:43.59s: badblocks -b 512 -c 8             1:40.16s: badblocks -b 4096 -c 1