24

Following instructions on this page, the page showed a dd command and a sync option at the end. The command is the following:

xzcat ~/ubuntu.img.xz | sudo dd of=/dev/sdX bs=32M sync

I know what is dd and how it works but I've never heard of or used the sync option with it and its manual page entry is like:

sync   pad every input block with NULs to  ibs-size;  when  used  with
       block or unblock, pad with spaces rather than NULs

What is NUL and why is it padded to ibs-size, and why bother padding the data blocks and use the sync option with the dd command? Why not keeping it simple and easy?

When I try to run sync with the command as it is with the appropriate location and values I get the following error:

dd: unrecognized operand ‘sync’
Try 'dd --help' for more information.
Braiam
  • 67,791
  • 32
  • 179
  • 269
Chinmaya B
  • 6,122
  • 7
  • 24
  • 43

4 Answers4

33

You are misreading the command. It's:

xzcat ~/ubuntu.img.xz | sudo dd of=/dev/sdX bs=32M 
sync

sync is a separate command, here. See man 1 sync:

NAME
       sync - flush file system buffers

SYNOPSIS
       sync [OPTION]

DESCRIPTION
       Force changed blocks to disk, update the super block.

Kinda obvious why it's used once you think of caching.

As for the dd option you read, it's a value for the conv option:

Each CONV symbol may be:
   ascii  from EBCDIC to ASCII
   ...
   sync   pad every input block with NULs  to  ibs-size;  when  used  with
          block or unblock, pad with spaces rather than NULs

If it was used (which it isn't), it would be used thus:

dd ... conv=sync
muru
  • 197,895
  • 55
  • 485
  • 740
  • 4
    Good answer, very detailed. Nice that you even describe the dd ... conv=sync argument. – Byte Commander Dec 06 '15 at 19:38
  • Don't let them taunt you! ;-) Good answer: you spotted the problem I didn't even see. >:-) +1 – Fabby Dec 06 '15 at 20:02
  • 1
    Also from the description of the conv=sync option it seems quite clear why it isn't used for this. At best it would do nothing, at worst it would corrupt the image during copying. – kasperd Dec 06 '15 at 23:22
  • 9
    Really the instructions should call for the dd option oflag=fsync ( which flushes the output dd has written ) instead of a sync command ( which flushes everything on the system. – psusi Dec 07 '15 at 00:10
  • 2
    This answer is misleading. With dd, sync is an argument to both conv and the iflag or oflag options. – Dan Loewenherz Aug 07 '16 at 20:16
  • @DanLoewenherz does that make a difference to the essence of the answer, that sync here is meant to be a command and not an option to dd or a value to a dd option? – muru Aug 08 '16 at 00:05
  • @psusi this is an incorrect combo oflag=fsync; fsync is a symbol for the conv operand. See info '(coreutils) dd invocation' as suggested in man dd - which has correct indentation. for fsync under the conv operand: Synchronize output data and metadata just before finishing. This forces a physical write of output data and metadata. The correct argument is conv=fsync. test: docker run -it --rm --name testdd ubuntu /bin/bash and then dd if=/dev/zero of=/dev/null conv=fsync count=1 which works. oflag=fsync throws invalid output flag: 'fsync' – Life5ign May 12 '23 at 03:22
7

Dd will read in data one block at a time (the block size is specified by the user). Dd may possibly read in a shorter block than the user specified, either at the end of the file or due to properties of the source device; this is called a partial record. By default, dd will then write out a block that's the same size as the amount that it read.

Dd will repeat this until the specified count is reached, or it sees eof on input, or error on input or output. When it finishes, dd reports the number of full and partial records it read and wrote.

This behavior is fine for copying an ordinary file within a filesystem or over a TCP network connection, since that's considered a stream of bytes. But other filesystem objects, such as raw DVDs and magnetic tape, require write sizes that are within certain ranges and are a round multiple of some amount (such as 512 or 2048 bytes). For example, if you have a disk image that is 255 tracks of 63 512-byte sectors, and want to write to a tape that requires a block size of 1024 bytes, you'd need to do something like

dd if=disk.img of=/dev/rmt0 bs=1k conv=sync

to make sure that dd doesn't try to write out a 512-byte block at the end. The shorter block at the end will be padded with zeroes or blanks. Padding with zeroes is a safe, common choice. The option for dd to pad with blanks is useful in a different situation - converting a file that has variable-length lines to one with fixed-length lines.

Mark Plotnick
  • 1,009
1

You can use sync option for output, which is included in dd comand, by oflag symbol:

xzcat ~/ubuntu.img.xz | sudo dd of=/dev/sdX bs=32M oflag=sync
Rudolf Dovičín
  • 197
0

I have found that the linux system, not dd, caches the buffers. Therefore, when the dd command has completed, I run sync as

sudo dd if=my.downloaded.iso of=/dev/sd(?) bs=1M && sync

I have not used any internal dd option to cause buffer flushing. Typically though, when I am creating a USB version of a distribution, I am the only one on the computer, so I have no qualms about my posted command doing harm. Since I run the sync as &&, sync is done with sudo rights and I am sure that the USB I create is complete.

Leslie Satenstein
  • 131