ImageMagick convert and low RAM

Question

Please consider the following command for the imagemagick package:

sudo convert -resize 460x200 /path/to/test1.jpg /path/to/test2.jpg

Now test1.jpg is a fairly large image (12.5MB), but not unheard of sizes, but the server I am working from only has 1G of memory (Amazon EC2 t2.micro if it helps)

No error is being thrown, and yet no test2.jpg is being created. I have tested this with smaller images and it works fine. I would have thought 1G if memory would be plenty to manage a resize, but perhaps not?

What are my options, do I have to get more memory or am I missing something?

Any suggestions are very welcome!

Requested Edits

strace output shows: +++ killed by SIGKILL +++ along with alot of commands that look normal (opening /usr/share/local and reading long strings, which I assume is image data)

free gives me:

         total       used       free     shared    buffers     cached
Mem:       1016292     278348     737944       6412        460      13356
-/+ buffers/cache:     264532     751760
Swap:            0          0          0

So it looks as if swap is not enabled, so I will look into creating a swapfile to see if this sorts my issues.

File gives me:

testprint10Mbv2.jpg: JPEG image data, EXIF standard

I would also paste the full /path/to/img but unfortunately it contains sensitive data (client email addresses etc)

Answer 1

I'm assuming you are running out of RAM. You can verify this with watch cat /proc/meminfo while your process is running.

You may have insufficient RAM and/or SWAP to accomplish your task.

Due to the low-ram condition, insure that swap is on with swapon -a and if no swap is setup on the system create a partition for swap on unused diskspace and enable it. This will likely solve the problem. There's a good answer by @Takkat on how to do that here If you don't have access to partitioning the server you can also use a swapfile for swap

Answer 2

The issue in the question was resolved by adding appropriate swap. For completeness let me give a summary of additional options we have when running Image Magick convert on huge files on low memory systems, or when there is no swap available, or the swap was too small.

The methods mentioned below are elaborated in detail in the Image Magick manual:

Really Massive Image Handling

---

Summary:

1. Limit memory usage with option -limit

   By doing so Image Magick will create a temporary file for image handling as soon as the given memory limits were exceeded. This needs write permission for Image Magick on the temporary file directory. We can give any path where the temporary file will be created in an environment variable MAGICK_TMPDIR. An example command may look similar to this:

       env MAGICK_TMPDIR=/tempdir nice -5 convert -limit memory 32 -limit map 32 largefile.jpg -resize 640x320 smallfile.png
   

   Working on disk rather than in RAM will slow down the processing speed considerably.

2. Work with "Memory Mapped Disk Files"

   Creating MPC files is resource demanding but it does not need so much resources to convert from MPC files. Therefore is may be put into consideration in case we need to convert the same source with different parameters several times. The workflow may then be similar to this:

   convert huge.jpg huge.mpc
   convert huge.mpc -resize 50% big.png
   convert huge.mpc -resize 20% small.png
   convert huge.mpc -resize 5% thumb.png
   

3. Work on small sections of an image using stream

   Using stream claims to only process a part of the source image without the need to load the whole image into memory.

   stream -map rgb -storage-type char -extract 600x400+1900+2900 image.png - | convert -depth 8 -size 600x400 rgb:- tile.png
   

   In above example the -extract option takes size and offset values as defined by the Image Magick geometry. We will have to stick the tiles back together for getting the scaled image as a whole. Unfortunately stream does not work for all image formats but is supposed to work fine on JPEG images.