0

I have a problem in decompressing file in Ubuntu. Here, I have huge amount of compressed file (test.tar.gz) which is nearly 1.2 TB and resultantly becomes 3 TB after decompressing.

I have only 4 TB and 1 TB SSD so that I cannot decompress the file in a single SSD.

So, my point is;

  1. How to split the (already) compressed file into several parts? (for example, splitting into 2 files, both of which are 600 GB)
  2. After that, I have to locate those files into different directories due to the amount problem. Then, How to decompress those files into a single directory even if they are located at the different directories?
David Oh
  • 1
  • "How to decompress splited files (from one compressed one)" that are possible. You need to merge the split files and can them decompress. – Rinzwind Mar 15 '23 at 08:03
  • What may work is to cat the parts of the archive to standard output, then have tar decompress the standard input – vanadium Mar 15 '23 at 13:18

2 Answers2

0

Use the split command :) If you want 2 files use -n with 2:

split test.tar.gz -n 2

Joining can be done with cat:

 cat x* > test2.tar.gz

will merge all files starting with x into test2.tar.gz Replace x* by the 2 files (and add directories).

Mind that you will need about double the space if all are on the same partition.

Rinzwind
  • 299,756
0

Split the tarball gzipped archive into two file and put one in the smaller SSD and the other into the bigger SSD; assuming your smaller SSD is mounted to /mnt/smallerSSD and the other SSD to /mnt/biggerSSD:

split -b 800M test.tar.gz test.tar.gz.part.
mv test.tar.gz.part.aa /mnt/smallerSSD
mv test.tar.gz.part.ab /mnt/biggerSSD

This will create two files test.tar.gz.part.aa and test.tar.gz.part.ab respectively of 800MB and about 400MB (sum is 1.2TB) and put them each in a different SSD. If you have less than 800MB of free disk space in your smaller SSD, use a smaller value.

Then, using cat and tar, proceed to extract the whole archive into the bigger SSD

cd /mnt/biggerSSD
cat /mnt/smallerSSD/test.tar.gz.part.aa /mnt/biggerSSD/test.tar.gz.part.ab | tar -xvz

This may not work if the free disk space is not enough; in that case you could use netcat/ssh to transfer (and then retrieve) some parts from the other devices in the local network, or else, you could use a more aggressive compression algorithm for your tarballs (e.g. bzip2) but this will result in longer compression/decompression times.

Marco
  • 1