Differences

This shows you the differences between two versions of the page.

--- software:tar-how-to-speed-up-tar-preventing-seek-especially-with-many-small-files [2024/05/07 16:47] – rik
+++ software:tar-how-to-speed-up-tar-preventing-seek-especially-with-many-small-files [2024/06/21 21:09] (current) – rik
@@ Line 32: / Line 32: @@
 <code bash>
 tar --zstd -cf - ./dir_to_archive | mbuffer -t -m500M -P80 -f -o arc.tar.zst
+</code>
+=====EVEN BETTER=====
+Sorting the files by inode, prevents seek while reading, for a further huge increment in speed while feeding the buffer.
+<code bash>
+# Sort
+find dir_to_archive -type f -print0 | xargs -0 stat --format='%i %n' | sort -n | cut -d' ' -f2- > filelist_sorted.txt
+# Archiving
+tar -cf - -T filelist_sorted.txt | mbuffer -t -m500M -P80 -f -o arc.tar
+# Or archiving with zstd: a bit of compression at basically the same speed
+tar --zstd -cf - -T filelist_sorted.txt | mbuffer -t -m500M -P80 -f -o arc.tar.zst
+</code>
+**Example**
+<code bash>
+find ./backup -type f -print0 | xargs -0 stat --format='%i %n' | sort -n | cut -d' ' -f2- > filelist_sorted.txt
+tar --zstd -cf - -T filelist_sorted.txt | mbuffer -t -m500M -P80 -f -o backup.tar.zst
 </code>