The automatic feature detection relies on the evaluation of the relevance of each pixel; this quantity is called `energy': the higher the energy of a pixel, the less likely it will be that such pixel will be directly involved in the rescaling. The energy function which the library uses can be customised; normally, edge detector filters are the best choices for this purpose.
There is a unified framework in the Liquid Rescale library for energy fucntions: briefly, each energy function used by the library receives the current position and image size as parameters, and is provided access to a square of values from the image, centered around the current position. This access is provided through an object which is called a "reading window", and the image pixels are not read directly, but rather they are accessed according to a "reader type", which means that the energy function can get e.g. the brightness of a pixel without needing to care for the underlying image representation, i.e. the image type or the colour depth.
The library provides a small set of very simple (yet normally effective) gradient-based energy functions, and a customisation framework for defining more functions.
The library has some builtin functions for energy evaluation; in order to set one of the builtin
functions in a LqrCarver
object this function is used:
LqrRetVal lqr_carver_set_energy_function_builtin( | LqrCarver* carver, |
LqrEnergyFunctionBuiltinType ef_ind) ; |
The currently available builtin functions which can be used as ef_ind
are:
LQR_EF_GRAD_XABS
absolute value of the brightness gradient in the direction of the rescaling (this is the default)
LQR_EF_GRAD_SUMABS
sum of absolute values of the brightness gradients in both directions
LQR_EF_GRAD_NORM
norm of the brightness gradient
LQR_EF_LUMA_GRAD_XABS
absolute value of the luma gradient in the direction of the rescaling
LQR_EF_LUMA_GRAD_SUMABS
sum of absolute values of the luma gradients in both directions
LQR_EF_LUMA_GRAD_NORM
norm of the luma gradient
LQR_EF_NULL
null
All of the above gradient functions have a radius of 1 pixel.
Custom energy functions for a LqrCarver
object are set using this function:
LqrRetVal lqr_carver_set_energy_function( | LqrCarver* carver, |
LqrEnergyFunc ef_func, | |
gint radius, | |
LqrEnergyReaderType reader_type, | |
gpointer extra_data) ; |
Here, ef_func
is the function (the prototype will be explained below),
radius
is used to set the size of the square region around each pixel which
is used to evaluate the energy for that pixel, reader_type
is used to set
which quantity is used for the energy computation (e.g. brightness, luma etc.) and
extra_data
is a (void) pointer which can be used to pass additional
parameters to the energy function.
The LqrEnergyReaderType
is an enum which can take these values (also
noted is the number of channels of the corresponging output):
LQR_ER_BRIGHTNESS
brightness (1 channel)
LQR_ER_LUMA
luma (1 channel)
LQR_ER_RGBA
RGBA (4 channels)
LQR_ER_CUSTOM
read the normalised image channels as they are (as many channels as the image has)
These readouts always return values beetween 0
and 1
.
Note that these readouts may have special meanings depending on the image type:
for LQR_GREY_IMAGE
, LQR_GREYA_IMAGE
and
LQR_CUSTOM_IMAGE
images, the LQR_ER_LUMA
readout will yield the same result as LQR_ER_BRIGHTNESS
for LQR_CUSTOM_IMAGE
images, the LQR_ER_BRIGHTNESS
readout will return the average pixel value (additive, i.e. if a black channel is
present the channel values will be inverted and multiplied by the black channel
inverse), multiplied by the alpha channel value.
for LQR_CUSTOM_IMAGE
images, the LQR_ER_RGBA
readout cannot be used: it will always return
0
The custom energy function must be declared like in the following sample declaration:
Example 2.5. Custom energy declaration
gfloat my_energy (gint x, gint y, gint width, gint height, LqrReadingWindow *rwindow, gpointer extra_data);
This function should return the energy at pixel x
, y
,
based on the knowledge of the current image size (obtained from width
and
height
) and the content of the image in a square around that pixel, which is
passed through the rwindow
reading window: in order to access this content,
this function must be used:
gdouble lqr_rwindow_read( | LqrReadingWindow * rwindow, |
gint x, | |
gint y, | |
channel channel) ; |
When reading out the content from a window, the x
and
y
parameters are relative to the window centre (which instead is given by the
x
, y
parameters passed to the custom energy function),
and they both range between -radius
and
radius
included (radius
being the same
one which is used in lqr_carver_set_energy_function
). The parameter
channel
specifies which channel to read out; depending on the
read_type
passed to lqr_carver_set_energy_function
the
range and meaning of this parameter will change: for the cases LQR_ER_BRIGHTNESS
and
LQR_ER_LUMA
it must be 0
, because the readout consists of a
single channel, for LQR_ER_RGBA
it must be between 0
and
3
(and then the readout will contain the RGBA information), while for
LQR_ER_CUSTOM
it must be one of the original image channels.
The rwindow
parameters can be read out from within a custom defined energy
function using these functions:
LqrEnergyReaderType lqr_rwindow_get_read_t( | LqrReadingWindow * rwindow) ; |
gint lqr_rwindow_get_radius( | LqrReadingWindow * rwindow) ; |
gint lqr_rwindow_get_channels( | LqrReadingWindow * rwindow) ; |
Following is an example of how a simple Sobel fliter can be defined and used within this framework:
Example 2.6. Custom energy definition and setup
/* definition */ gfloat sobel(gint x, gint y, gint width, gint height, LqrReadingWindow *rw, gpointer extra_data) { gint i, j; gdouble ex = 0; gdouble ey = 0; gdouble k[3][3] = {{0.125, 0.25, 0.125}, {0, 0, 0}, {-0.125, -0.25, -0.125}}; for (i = -1; i <=1; i++) { for (j = -1; j <=1; j++) { ex += k[i + 1][j + 1] * lqr_rwindow_read(rw, i, j, 0); ey += k[j + 1][i + 1] * lqr_rwindow_read(rw, i, j, 0); } } return (gfloat) sqrt(ex * ex + ey * ey); } /* usage */ lqr_carver_set_energy_function (carver, sobel, 1, LQR_ER_BRIGHTNESS, NULL);
In the above sobel
function it is assumed that the radius is 1 and that the
readout will consist of a single channel. Furthermore, no boundary checking is performed, and
therefore the parameters x
, y
, width
and height
are not used. Also, no extra
parameters are passed to the function.
The function lqr_rwindow_read
returns 0
when the
requested value is out of the image boundary or beyond the reading window radius.
The energy functions are called over the transposed image in case of vertical scaling,
therefore, if they are asymmetrical, the result will be different depending on the scaling
orientation (this is the case for example for the LQR_EF_GRAD_XABS
builtin
function).
The energy function output should be normalised in order to be comparable with the builtin
functions, otherwise the scale of the bias would be different depending on the energy function
used. This is the reason why in the Sobel filter written above the kernel
k
is scaled so that the sum of the absolute values of the matrix is equal
to 1
.
In actual code, the call to lqr_carver_set_energy_function
should be
protected to test its return value.
The energy can be read out with three similar functions, all of which fill a buffer provided by the user, but in different ways: two of them are suitable for plotting, while the third one can be used to retreive the true values of the energy, as used internally.
The first function can be used to get a buffer of normalised values, and it has a simple syntax:
LqrRetVal lqr_carver_get_energy( | LqrCarver * carver, |
gfloat * buffer, | |
gint orientation) ; |
This function writes the the energy values at the time of calling to the given
buffer
, ordered by row and then by column. All values range between
0
and 1
. The buffer must be allocated by the user before
calling it.
The orientation
parameter determines which orientation will be used when
computing the energy: 0
means horizontal, 1
means vertical.
The carver
need not be initialised; if a bias was added to it (see the
Adding a bias section), its effect is included in the output.
The above function does not return the true energy which is used internally, because that would not be suitable for plotting, since the energy range is arbitrary (only the energy differences are meaningful, and there is no upper nor lower bound to the values). In order to get the true energy values used internally, this function must be used instead (with the same syntax):
LqrRetVal lqr_carver_get_true_energy( | LqrCarver * carver, |
gfloat * buffer, | |
gint orientation) ; |
The third function can be used to fill directly an image buffer:
LqrRetVal lqr_carver_get_energy_image( | LqrCarver * carver, |
void * buffer, | |
gint orientation, | |
LqrColDepth col_depth, | |
LqrImageType image_type) ; |
In this case, the buffer must be passed as void*, while the
col_depth
and image_type
are used to specify the
colour depth and the image type, in the same way as is done when creating an LqrCarver
object (see
Carver objcet creation and the
Choosing the image type sections). The only restriction is that it
is not possible to ask for LQR_CUSTOM_IMAGE
image types.
For any choice of the parameters, the buffer
will hold a greyscale image,
ranging from black (lowest energy) to white (highest energy). The opacity (alpha) will be set to
1
, if present. All the information will be stored in the black channel in
LQR_CMYK_IMAGE
and LQR_CMYKA_IMAGE
image types.
Calling the function lqr_carver_get_energy_image
with LQR_COLDEPTH_32F
and
LQR_GREY_IMAGE
arguments will yield the same result as calling the function
lqr_carver_get_energy
, but the latter is slightly more efficient.