What's New in 7.28
- libvips rewrite
- About a third of libvips has been rewritten with the new framework that's been created for vips8. New operations are faster, more flexible and easier to use. There's full compatibility with the old interface as well. See below for some examples.
- OpenSlide support
- libvips now supports OpenSlide, a reader library for biological slides. Contributed by Benjamin Gilbert.
- Better file format support
- Improvements to jpeg, png and tiff handling let you manipulate EXIF tags, support better jpeg-in-tiff compression and support a wider range of png images.
- Sequential mode read
- A new image read mode lets you perform some operations (like thumbnailing) on some image formats much more quickly. See below.
- Operation cache
- libvips now keeps a cache of recent operations --- when you perform a calculation, it'll reuse old results from the cache when it can. This can produce a huge speedup in complex programs. You can also turn on cache tracing, very handy for debugging, see below for details.
- Minor nip2 improvements
- There's a new image header browser, fixes to help the Unity interface on Ubuntu, a new right-click menu for images and some more Matrix operations. See below.
- Better Windows build
- The Windows version has been built with a new version of the GTK toolkit that should be quite a bit better. There are improvements to appearance, speed, file dialogs, drag and drop and copy paste support.
Plus the usual minor speed-ups, portability improvements, enhancements and bug fixes. What's New in 7.26 is still there if you're interested.
The right-button on image thumbnails now has a Header item.
This takes you directly to the image header display. Header fields are now shown much more clearly, and there's a new search box at the upper-right. This lets you filter the fields which are shown. This is handy for dealing with formats which have hundreds of fields, like FITS or DICOM.
There's a new right-button menu on image backgrounds. This can be quicker than using the top menus.
Finally, a new set of Matrix menu items let you sort matrices and generate series.
In 7.26 we introduced a new core for libvips. In this version we've started rewriting libvips operators to use this new system. A set of compatibility functions now implement the old vips7 interface on top of these new operators.
One of the big new features is optional output arguments. For example,
VipsMin is an operation that finds the minimum value of an image. You can run it from the command-line like this:
$ vips min fred.jpg 12
Very similar to the old
im_min. But now there are also optional output arguments to get the position of the minimum:
$ vips min --x --y fred.jpg 12 2345 245
Run with no arguments to get a handy help display:
$ vips min usage: min in out where: in - Input image, input VipsImage out - Output value, output gdouble optional arguments: y - Vertical position of minimum, output gint x - Horizontal position of minimum, output gint
>> m = im.min() >> print m 12 >> m, x, y = im.min(x = True, y = True) VipsImage *fred; double m; int x, y; vips_min (fred, &m, "x", &x, "y", &y, NULL);
The old vips7 interface is still there, just use the "im_" prefix to get that.
$ vips im_min usage: vips im_min in where: in is of type "image" prints: value of type "double" minimum value of image, from package "arithmetic" flags: (PIO function) (no coordinate transformation) (area operation) (result can be cached)
VipsJoin has one of the fanciest ones. At a minimum it's:
$ vips join left.v right.v out.v horizontal
to join two images left-right, but there are also extra optional arguments, for example:
$ vips join --expand --background 0,255,0 --align centre --shim 50 left.v right.v out.v horizontal
to join left-right with a 50 pixel gap, the output expanded to hold all of both left and right, and the background painted green.
libvips now keeps a cache of recent operations. When you run something, it checks the cache and reuses the old result if it can. This can produce a huge speedup for complex programs.
There are a set of command-line switches to control how many operations are cached, how much memory to allocate to the cache and how many file descriptors you are prepared to give it. You can also trace cache behaviour, which can be handy for debugging.
For example, here's a tiny Python script to find the maximum of an image using the new vips8 interface:
#!/usr/bin/python import sys from vips8 import vips # turn on the cache trace from gi.repository import Vips Vips.cache_set_trace(True) a = vips.Image(sys.argv) print a.max() print a.max()
When you run it, the
cache_set_trace(True) enables cache tracing and you can see what operations actually get run:
$ ./try3.py ~/pics/Gugg_coloured.jpg vips cache: miss 0x21a80c0 jpegload "/home/john/pics/Gugg_coloured.jpg" VipsForeignLoadJpegFile (0x21a80c0) vips cache: miss 0x21a81e0 max ((vips8+vips+Image*) 0x21a7000) VipsMax (0x21a81e0) vips cache: miss 0x21c1650 sequential ((VipsImage*) 0x21a7730) VipsSequential (0x21c1650) vips cache: miss 0x21a86f0 tilecache ((VipsImage*) 0x21a78a0) VipsTileCache (0x21a86f0) 255.0 vips cache: hit 0x21a81e0 max ((vips8+vips+Image*) 0x21a7000) VipsMax (0x21a8660) 255.0
So the first
max() causes four operations to run (jpegload, max, sequential, tilecache), the second just pulls the 255 from the operation cache. The command-line switch
--vips-cache-trace does the same for command-line programs.
New file format system
In vips7 you specify file formats and give them options like this:
$ vips im_vips2tiff myfile.v newfile.tiff:jpeg
meaning load the file
myfile.v, then write it to
newfile.tiff in TIFF format with jpeg compression.
The vips8 equivalent is called
tiffsave and it uses all the optional argument features of the new vips8 operations. If you run it with no arguments you get a nice help display:
$ vips tiffsave usage: tiffsave in filename where: in - Image to save, input VipsImage filename - Filename to save to, input gchararray optional arguments: compression - Compression for this file, input VipsForeignTiffCompression Q - Q factor, input gint predictor - Compression prediction, input VipsForeignTiffPredictor profile - ICC profile to embed, input gchararray tile - Write a tiled tiff, input gboolean tile-width - Tile width in pixels, input gint tile-height - Tile height in pixels, input gint pyramid - Write a pyramidal tiff, input gboolean squash - Squash images down to 1 bit, input gboolean resunit - Resolution unit, input VipsForeignTiffResunit bigtiff - Write a bigtiff image, input gboolean
and you can do the equivalent of the old vips7 command above like this:
$ vips tiffsave myfile.v newfile.tiff --compression jpeg
You can also embed options in the file name using the standard vips8 syntax. For example:
$ vips invert fred.jpg jim.tif[compression=jpeg]
Which will load
fred.jpg, invert it, then save as a jpeg-compressed tiff.
The old vips7 system is still there and working, though it's now just a thin skin over the new system.
Sequential mode read
Many image formats only really support sequential reading. For example, libpng provides
png_read_row(), a function which reads the next line of pixels from a file. You call it once for each line in the image to get every pixel.
However, many operations require random access to pixels. For example, a 90-degree rotate will need to read a column of pixels for every output line it writes. If it just used
png_read_row() it would need to decompress the input image many, many times in order to create the output.
To prevent this, libvips will first decompress the entire image to a large temporary area and then process from that. This temporary area is in memory for small files or on disc for large files.
Not all operations need random access to their source pixels. For example, thumbnailing, the process of shrinking images for display, can work strictly top-to-bottom. To help speed up operations of this type, libvips has a new hint that you can give to read operations to indicate that you only need sequential access.
$ time vips --vips-leak copy wtc.jpg[sequential] wtc.tif real 0m4.903s user 0m1.384s sys 0m0.512s memory: high-water mark 70.73 MB
wtc.jpg is a 10,000 by 10,000 pixel RGB image and
--vips-leak makes libvips print peak memory use)
[sequential] hint means that we will only need top-to-bottom access to the pixels in wtc.jpg. When libvips opens this image it will not decompress to a temporary buffer but, instead, stream pixels from the jpeg decompressor directly through the operation. This saves first writing and then reading the 300 MB temporary area.
The sequential mode reader keeps a few hundred lines behind the current read point in a cache so it can handle some small amount of non-sequential access. If you try something that's very non-sequential after giving the
[sequential] hint, you get an error.
$ time vips flip wtc.png[sequential] wtc.tif vertical VipsSequential: non-sequential read --- at position 0 in file, but position 9344 requested
[sequential] hint you get the current libvips behaviour, which is noticeably slower and of course makes much more disc traffic:
$ time vips copy wtc.jpg wtc.tif real 0m6.004s user 0m1.588s sys 0m1.208s memory: high-water mark 77.46 MB
The jpeg, stripped tiff and png readers now support sequential mode read.
The libvips thumbnailing utility, vipsthumbnail, now turns on sequential mode for you, so you get this nice behaviour automatically.
You can see a good speedup when building large pyramids:
$ time vips copy world.topo.bathy.200405.3x21600x21600.A1.jpg A1-pyr.tif[compression=jpeg,tile,pyramid] real 1m12.996s user 0m22.845s sys 0m4.552s $ time vips copy world.topo.bathy.200405.3x21600x21600.A1.jpg[sequential] A1-pyr.tif[compression=jpeg,tile,pyramid] real 0m24.218s user 0m22.277s sys 0m1.172s