Add lensfun filter

Lensfun is a library that applies lens correction to an image using a database of cameras/lenses (you provide the camera and lens models, and it uses the corresponding database entry's parameters to apply lens correction). It is licensed under LGPL3. The lensfun filter utilizes the lensfun library to apply lens correction to videos as well as images. This filter was created out of necessity since I wanted to apply lens correction to a video and the lenscorrection filter did not work for me. While this filter requires little info from the user to apply lens correction, the flaw is that lensfun is intended to be used on indvidual images. When used on a video, the parameters such as focal length is constant, so lens correction may fail on videos where the camera's focal length changes (zooming in or out via zoom lens). To use this filter correctly on videos where such parameters change, timeline editing may be used since this filter supports it. Note that valgrind shows a small memory leak which is not from this filter but from the lensfun library (memory is allocated when loading the lensfun database but it somehow isn't deallocated even during cleanup; it is briefly created in the init function of the filter, and destroyed before the init function returns). This may have been fixed by the latest commit in the lensfun repository; the current latest release of lensfun is almost 3 years ago. Bi-Linear interpolation is used by default as lanczos interpolation shows more artifacts in the corrected image in my tests. The lanczos interpolation is derived from lenstool's implementation of lanczos interpolation. Lenstool is an app within the lensfun repository which is licensed under GPL3. v2 of this patch fixes license notice in libavfilter/vf_lensfun.c v3 of this patch fixes code style and dependency to gplv3 (thanks to Paul B Mahol for pointing out the mentioned issues). v4 of this patch fixes more code style issues that were missed in v3. v5 of this patch adds line breaks to some of the documentation in doc/filters.texi (thanks to Gyan Doshi for pointing out the issue). v6 of this patch fixes more problems (thanks to Moritz Barsnick for pointing them out). v7 of this patch fixes use of sqrt() (changed to sqrtf(); thanks to Moritz Barsnick for pointing this out). Also should be rebased off of latest master branch commits at this point. Signed-off-by: Stephen Seo <seo.disparate@gmail.com>
FFmpeg · Jul 15, 2018 · 0ea2012 · 0ea2012
1 parent c5329d6
commit 0ea2012
Show file tree

Hide file tree

Showing 5 changed files with 672 additions and 0 deletions.
diff --git a/configure b/configure
@@ -239,6 +239,7 @@ External library support:
   --enable-libilbc         enable iLBC de/encoding via libilbc [no]
   --enable-libjack         enable JACK audio sound server [no]
   --enable-libkvazaar      enable HEVC encoding via libkvazaar [no]
+  --enable-liblensfun      enable lensfun lens correction [no]
   --enable-libmodplug      enable ModPlug via libmodplug [no]
   --enable-libmp3lame      enable MP3 encoding via libmp3lame [no]
   --enable-libopencore-amrnb enable AMR-NB de/encoding via libopencore-amrnb [no]
@@ -1656,6 +1657,7 @@ EXTERNAL_LIBRARY_NONFREE_LIST="
 
 EXTERNAL_LIBRARY_VERSION3_LIST="
     gmp
+    liblensfun
     libopencore_amrnb
     libopencore_amrwb
     libvmaf
@@ -3353,6 +3355,7 @@ hqdn3d_filter_deps="gpl"
 interlace_filter_deps="gpl"
 kerndeint_filter_deps="gpl"
 ladspa_filter_deps="ladspa libdl"
+lensfun_filter_deps="liblensfun version3"
 lv2_filter_deps="lv2"
 mcdeint_filter_deps="avcodec gpl"
 movie_filter_deps="avcodec avformat"
@@ -6023,6 +6026,7 @@ enabled libgsm            && { for gsm_hdr in "gsm.h" "gsm/gsm.h"; do
                                done || die "ERROR: libgsm not found"; }
 enabled libilbc           && require libilbc ilbc.h WebRtcIlbcfix_InitDecode -lilbc $pthreads_extralibs
 enabled libkvazaar        && require_pkg_config libkvazaar "kvazaar >= 0.8.1" kvazaar.h kvz_api_get
+enabled liblensfun        && require_pkg_config liblensfun lensfun lensfun.h lf_db_new
 # While it may appear that require is being used as a pkg-config
 # fallback for libmfx, it is actually being used to detect a different
 # installation route altogether.  If libmfx is installed via the Intel

diff --git a/doc/filters.texi b/doc/filters.texi
@@ -10700,6 +10700,124 @@ The formula that generates the correction is:
 where @var{r_0} is halve of the image diagonal and @var{r_src} and @var{r_tgt} are the
 distances from the focal point in the source and target images, respectively.
 
+@section lensfun
+
+Apply lens correction via the lensfun library (@url{http://lensfun.sourceforge.net/}).
+
+The @code{lensfun} filter requires the camera make, camera model, and lens model
+to apply the lens correction. The filter will load the lensfun database and
+query it to find the corresponding camera and lens entries in the database. As
+long as these entries can be found with the given options, the filter can
+perform corrections on frames. Note that incomplete strings will result in the
+filter choosing the best match with the given options, and the filter will
+output the chosen camera and lens models (logged with level "info"). You must
+provide the make, camera model, and lens model as they are required.
+
+The filter accepts the following options:
+
+@table @option
+@item make
+The make of the camera (for example, "Canon"). This option is required.
+
+@item model
+The model of the camera (for example, "Canon EOS 100D"). This option is
+required.
+
+@item lens_model
+The model of the lens (for example, "Canon EF-S 18-55mm f/3.5-5.6 IS STM"). This
+option is required.
+
+@item mode
+The type of correction to apply. The following values are valid options:
+
+@table @samp
+@item vignetting
+Enables fixing lens vignetting.
+
+@item geometry
+Enables fixing lens geometry. This is the default.
+
+@item subpixel
+Enables fixing chromatic aberrations.
+
+@item vig_geo
+Enables fixing lens vignetting and lens geometry.
+
+@item vig_subpixel
+Enables fixing lens vignetting and chromatic aberrations.
+
+@item distortion
+Enables fixing both lens geometry and chromatic aberrations.
+
+@item all
+Enables all possible corrections.
+
+@end table
+@item focal_length
+The focal length of the image/video (zoom; expected constant for video). For
+example, a 18--55mm lens has focal length range of [18--55], so a value in that
+range should be chosen when using that lens. Default 18.
+
+@item aperture
+The aperture of the image/video (expected constant for video). Note that
+aperture is only used for vignetting correction. Default 3.5.
+
+@item focus_distance
+The focus distance of the image/video (expected constant for video). Note that
+focus distance is only used for vignetting and only slightly affects the
+vignetting correction process. If unknown, leave it at the default value (which
+is 1000).
+
+@item target_geometry
+The target geometry of the output image/video. The following values are valid
+options:
+
+@table @samp
+@item rectilinear (default)
+@item fisheye
+@item panoramic
+@item equirectangular
+@item fisheye_orthographic
+@item fisheye_stereographic
+@item fisheye_equisolid
+@item fisheye_thoby
+@end table
+@item reverse
+Apply the reverse of image correction (instead of correcting distortion, apply
+it).
+
+@item interpolation
+The type of interpolation used when correcting distortion. The following values
+are valid options:
+
+@table @samp
+@item nearest
+@item linear (default)
+@item lanczos
+@end table
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Apply lens correction with make "Canon", camera model "Canon EOS 100D", and lens
+model "Canon EF-S 18-55mm f/3.5-5.6 IS STM" with focal length of "18" and
+aperture of "8.0".
+
+@example
+ffmpeg -i input.mov -vf lensfun=make=Canon:model="Canon EOS 100D":lens_model="Canon EF-S 18-55mm f/3.5-5.6 IS STM":focal_length=18:aperture=8 -c:v h264 -b:v 8000k output.mov
+@end example
+
+@item
+Apply the same as before, but only for the first 5 seconds of video.
+
+@example
+ffmpeg -i input.mov -vf lensfun=make=Canon:model="Canon EOS 100D":lens_model="Canon EF-S 18-55mm f/3.5-5.6 IS STM":focal_length=18:aperture=8:enable='lte(t\,5)' -c:v h264 -b:v 8000k output.mov
+@end example
+
+@end itemize
+
 @section libvmaf
 
 Obtain the VMAF (Video Multi-Method Assessment Fusion)

diff --git a/libavfilter/Makefile b/libavfilter/Makefile
@@ -248,6 +248,7 @@ OBJS-$(CONFIG_INTERLACE_FILTER)              += vf_tinterlace.o
 OBJS-$(CONFIG_INTERLEAVE_FILTER)             += f_interleave.o
 OBJS-$(CONFIG_KERNDEINT_FILTER)              += vf_kerndeint.o
 OBJS-$(CONFIG_LENSCORRECTION_FILTER)         += vf_lenscorrection.o
+OBJS-$(CONFIG_LENSFUN_FILTER)                += vf_lensfun.o
 OBJS-$(CONFIG_LIBVMAF_FILTER)                += vf_libvmaf.o framesync.o
 OBJS-$(CONFIG_LIMITER_FILTER)                += vf_limiter.o
 OBJS-$(CONFIG_LOOP_FILTER)                   += f_loop.o

diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
@@ -237,6 +237,7 @@ extern AVFilter ff_vf_interlace;
 extern AVFilter ff_vf_interleave;
 extern AVFilter ff_vf_kerndeint;
 extern AVFilter ff_vf_lenscorrection;
+extern AVFilter ff_vf_lensfun;
 extern AVFilter ff_vf_libvmaf;
 extern AVFilter ff_vf_limiter;
 extern AVFilter ff_vf_loop;