HACKING - external/liblouis-github - Git at Google

 This HACKING file describes the development environment.  	-*- org -*-

   Copyright (C) 2008, 2009, 2011 ViewPlus Technologies, Inc. and Abilitiessoft, Inc.
   Copyright (C) 2012, 2013, 2014,2015 Swiss Library for the Blind, Visually Impaired and Print Disabled

   Copying and distribution of this file, with or without modification,
   are permitted in any medium without royalty provided the copyright
   notice and this notice are preserved. This file is offered as-is,
   without any warranty.

 This file attempts to describe the maintainer-specific notes to follow
 when hacking liblouis.

 * Developing
 ** Where to get it
 The development sources are available through git at github.com:

   https://github.com/liblouis/liblouis

 ** Build requirements
 To build Automake, Autoconf, and Libtool are used. If you are getting
 the sources from git (or change configure.ac), you'll need to have
 these tools installed to (re)build. Optionally (if you want to
 generate man pages) you'll also need help2man. All of these programs
 are available from ftp://ftp.gnu.org/gnu.

 If you want to run the YAML based test suite you will have to install
 ~libyaml~.

 On Mac OS, the programs can be optained with Homebrew (http://brew.sh):

   brew install automake libtool pkg-config texinfo

 Note that if you are using Homebrew to install liblouis (see below),
 the build dependencies are installed automatically.

 ** Gnulib
 Gnulib (http://www.gnu.org/software/gnulib) is used to provide
 portable basic functionality to programs and libraries. We use two
 instances of gnulib, one to provide portable functions such as
 ~malloc~, ~strndup~, etc to the library and another one to provide
 portable functionality such as ~getopt~, ~progname~ or
 ~version-etc~ to the tools.

 The first time invocation to import gnulib for the library was

   gnulib-tool --add-import --lib=libgnu --source-base=gnulib \
               --m4-base=gnulib/m4 --aux-dir=build-aux --libtool \
 	      --macro-prefix=gl --no-vc-files \
 	      malloc-gnu realloc-gnu strndup

 and for the tools

   gnulib-tool --add-import --lib=libgnutools --source-base=tools/gnulib \
               --m4-base=tools/gnulib/m4 --aux-dir=build-aux --libtool \
 	      --macro-prefix=gl_tools --no-vc-files \
 	      getopt-gnu malloc-gnu progname version-etc

 More modules might have been added since. The currently-used gnulib
 modules and other gnulib information are recorded in
 ~gnulib/m4/gnulib-cache.m4~ and ~tools/gnulib/m4/gnulib-cache.m4~.

 If you want to update from the current gnulib, install gnulib, and
 then run the following commands in the top-level directory.

   gnulib-tool --add-import --lib=libgnu --source-base=gnulib \
               --m4-base=gnulib/m4 --aux-dir=build-aux --libtool \
 	      --macro-prefix=gl --no-vc-files
   gnulib-tool --add-import --lib=libgnutools --source-base=tools/gnulib \
               --m4-base=tools/gnulib/m4 --aux-dir=build-aux --libtool \
 	      --macro-prefix=gl_tools --no-vc-files

 ** How to build
 After getting the sources from git, with

   git clone https://github.com/liblouis/liblouis.git

 and installing the tools above, change to the liblouis directory and
 and bootstrap the project with the following command

   ./autogen.sh

 to do a fresh build. Then run configure as usual:

   ./configure

 You have the choice to compile liblouis for either 16- or 32-bit
 Unicode. By default it is compiled for the former. To get 32-bit
 Unicode run configure with --enable-ucs4 .

 After running configure run "make" and then "make install". You must
 have root privileges for the installation step.

 ** Install with Homebrew
 Homebrew (http://brew.sh) is a package manager for Mac OS X that
 installs software from source. There is nothing special about the
 installation process in the sense that under the hood it happens
 exactly as described above, with the only difference that Homebrew
 automates it completely.

 First, use the ~brew tap~ command to add the repository that includes
 the liblouis formula:

   brew tap liblouis/liblouis

 Now you are ready to install liblouis:

   brew install liblouis

 ** Docker
 Docker (https://www.docker.com) can be useful both for creating a
 development environment for liblouis, and for shipping the
 application.

 Setting up a developer environment can take long and can be
 problematic especially for Windows. Thanks to Docker we can set up the
 environment for you, we can easily distribute it as an image, which
 can be run by anybody and will behave exactly the same for everybody.

 Docker images of liblouis are being built automatically each time
 something changes in the code (see https://registry.hub.docker.com/repos/liblouis).
 In order to use them, first get Docker at
 http://docs.docker.com/introduction/get-docker. Download the latest
 liblouis image with:

   docker pull liblouis/liblouis

 Then, enter the development environment by running the image in a
 Docker container:

   docker run -it liblouis/liblouis bash

 Local files and directories can be "mounted" inside the container, in
 order to make it easier to edit files and to persist changes across
 runs. For example, to use local table files:

   docker run -it -v $(pwd)/tables:/tmp/liblouis/tables liblouis/liblouis bash

 See the Docker documentation for more info.

 The same Docker image can be used as a development environment and as
 the application itself. For example, to run the lou_translate tool
 from inside a Docker container:

   docker run -it liblouis/liblouis lou_translate en-us-g1.ctb

 To rebuild the image yourself, run the following command in the root
 directory of the liblouis source:

   docker build -t liblouis/liblouis .

 A .dockerignore file is required if you want to compile the source both on te host and in the Docker
 container. The .dockerignore file can be updated from .gitignore with:

   make .dockerignore

 ** How to run tests
 Tests are run with

   make check

 ** How to debug
 First you have to build liblouis with debugging info enabled.

   $ ./configure CFLAGS='-g -O0 -Wall -Wextra'
   $ make

 Starting the programs under the tools directory within gdb is a little
 tricky as they are linked with libtool. See the info page of libtool
 for more information. To start lou_checktable for table wiskunde.ctb
 for example you'd have to issue the following commands:

   $ libtool --mode=execute gdb ./tools/lou_checktable
   (gdb) run tables/wiskunde.ctb

 ** How to find memory leaks
 Valgrind is a tool that can be used to find memory errors. It is
 recommended that you compile liblouis without any optimizations and
 with all warnings enabled before running it through Valgrind:

   $ ./configure CFLAGS='-g -O0 -Wall'
   $ make

 Then use Valgrind to analyze liblouis. For example you can run
 lou_translate trough Valgrind:

   $ libtool --mode=execute valgrind -v --tool=memcheck \
     --leak-check=full --leak-resolution=high --log-file=valgrind.log \
     ./tools/lou_translate en-us-g2.ctb

 Type a few words at the prompt, check translation and terminate
 lou_translate. Now open the file valgrind.log and see if there are any
 memory leaks reported.

 You can also just run lou_checktable for example:

   $ libtool --mode=execute valgrind -v --tool=memcheck \
     --leak-check=full --leak-resolution=high --log-file=valgrind.log \
     ./tools/lou_checktable tables/nl-BE-g0.utb

 Again open valgrind.log to see if any memory leaks were reported.

 For the full experience run lou_allround under Valgrind:

   $ libtool --mode=execute valgrind -v --tool=memcheck \
     --leak-check=full --show-reachable=yes \
     --leak-resolution=high --track-origins=yes \
     --log-file=valgrind.log ./tools/lou_allround

 ** How to analyze performance
 Gprof helps you analyze the performance of programs. You have to
 compile liblouis as follows:

   $ ./configure --disable-shared
   $ make clean all CFLAGS='-g -O0 -pg' LDFLAGS='-all-static'

 Then translate some stuff with a large table:

   $ ./tools/lou_translate tests/tables/large.ctb

 Finally look at the call profile:

   $ libtool --mode=execute gprof ./tools/lou_translate gmon.out

 ** How to build for Windows
 See the README.windows file and the windows subdirectory.

 *** How to cross-compile for Windows
 To compile for win32, use the MinGW win32 cross-compiler as shown
 below. Use the prefix option to install the binaries to a temporary
 place where you can create a zip file. The LDFLAGS='-all-static'
 ensures that libgcc is linked in statically. Otherwise the users need
 to have libgcc_s_sjlj-1.dll.

 Some users want the dlls unversioned. To achieve that add
 '-avoid-version' to LDFLAGS.

   ./configure --build=i686-pc-linux-gnu --host=i686-w64-mingw32 --prefix=/tmp/liblouis-mingw32
   make LDFLAGS='-avoid-version -Xcompiler -static-libgcc'
   make install
   zip -r liblouis-mingw32msvc.zip /tmp/liblouis-mingw32

 To compile for win64, use the MinGW-w64 cross-compiler:

   ./configure --build=i686-pc-linux-gnu --host=x86_64-w64-mingw32 --prefix=/tmp/liblouis-w64-mingw32
   make LDFLAGS='-avoid-version -Xcompiler -static-libgcc'
   make install
   zip -r liblouis-w64-mingw32.zip /tmp/liblouis-w64-mingw32

 Two makefile rules have been provided to do this automatically for you. Docker will be used if the
 right cross-compiler is not installed. To compile for win32 using the MinGW cross-compiler, run make
 distwin32. This will produce a ZIP file called liblouis-<version>-win32.zip. To compile for win64
 using the MinGW-w64 cross-compiler, run make distwin64. This will produce a ZIP file called
 liblouis-<version>-win64.zip.

 *** TODO How to build for Windows using Cygwin
 (possibly use a Vagrantfile as demonstration + explain that Cygwin
 binaries can not be used outside the Cygwin environment)

 *** TODO How to build for Windows using MinGW
 (possibly use a Vagrant file as demonstration)


 * Release Procedure
 These steps describe what a maintainer does to make a release; they
 are not needed for ordinary patch submission.

 ** Update the NEWS entry
 If any new tables were added or renamed please note them with their
 name in the NEWS entry. This is usefull for projects like NVDA.

 ** Set the version number
 Update the version number in NEWS (with version, date, and release
 type), ChangeLog, configure.ac and windows/include/config.h.

 Don't forget to update the libtool versioning info in configure.ac,
 i.e. LIBLOUIS_REVISION and possibly LIBLOUIS_CURRENT and LIBLOUIS_AGE.

 ** Commit and tag
 Commit the changes and tag this version

   git tag -s v2.6.0 -m "Release 2.6.0"
   git push origin v2.6.0

 If you know the exact version number that needs to be tagged use

   git tag -s v2.6.0 -m "Release 2.6.0" <commit>
   git push origin v2.6.0

 ** Make the release
 Check out a clean copy in a different directory, like /tmp. Run
 autogen.sh and configure with no special prefixes. Run make distcheck.
 This will make sure that all needed files are present, and do a
 general sanity check. Run make dist. This will produce a tarball.

   ./autogen.sh && ./configure && make && make distcheck && make dist

 ** Upload
 Add the tarball to the github liblouis releases page, i.e. add it
 under https://github.com/liblouis/liblouis/releases with the specific
 release and add a link to it in $WEBSITE/downloads/index.md. See below
 for instructions on how to update the web site.

 ** Online documentation
 The online documentation is part of the liblouis web site. To add it to the
 site simply copy doc/liblouis.html to $WEBSITE/documentation/liblouis.html.
 Make sure you add the proper YAML front matter. Again see below for
 instructions on how to update the web site.

 ** Web site maintenance
 The liblouis web site at liblouis.org is maintained with the help of
 github pages (https://pages.github.com/). To edit the site just check
 out the repo at https://github.com/liblouis/liblouis.github.io. You'll
 need to know a few things about Jekyll (http://jekyllrb.com/) and
 markdown, the markup that is used to edit the content. In order to
 update the site simply edit, commit and push.

 For the new release update the project web site.

 - Add a post containing the current NEWS to the _posts directory and
 - add the download artifacts to the download page

 ** Announce
 Send an announcement to the liblouis list
 liblouis-liblouisxml@freelists.org. See ANNOUNCEMENT for an example.
	This HACKING file describes the development environment. -- org --

	Copyright (C) 2008, 2009, 2011 ViewPlus Technologies, Inc. and Abilitiessoft, Inc.
	Copyright (C) 2012, 2013, 2014,2015 Swiss Library for the Blind, Visually Impaired and Print Disabled

	Copying and distribution of this file, with or without modification,
	are permitted in any medium without royalty provided the copyright
	notice and this notice are preserved. This file is offered as-is,
	without any warranty.

	This file attempts to describe the maintainer-specific notes to follow
	when hacking liblouis.

	* Developing
	** Where to get it
	The development sources are available through git at github.com:

	https://github.com/liblouis/liblouis

	** Build requirements
	To build Automake, Autoconf, and Libtool are used. If you are getting
	the sources from git (or change configure.ac), you'll need to have
	these tools installed to (re)build. Optionally (if you want to
	generate man pages) you'll also need help2man. All of these programs
	are available from ftp://ftp.gnu.org/gnu.

	If you want to run the YAML based test suite you will have to install
	~libyaml~.

	On Mac OS, the programs can be optained with Homebrew (http://brew.sh):

	brew install automake libtool pkg-config texinfo

	Note that if you are using Homebrew to install liblouis (see below),
	the build dependencies are installed automatically.

	** Gnulib
	Gnulib (http://www.gnu.org/software/gnulib) is used to provide
	portable basic functionality to programs and libraries. We use two
	instances of gnulib, one to provide portable functions such as
	~malloc~, ~strndup~, etc to the library and another one to provide
	portable functionality such as ~getopt~, ~progname~ or
	~version-etc~ to the tools.

	The first time invocation to import gnulib for the library was

	gnulib-tool --add-import --lib=libgnu --source-base=gnulib \
	--m4-base=gnulib/m4 --aux-dir=build-aux --libtool \
	--macro-prefix=gl --no-vc-files \
	malloc-gnu realloc-gnu strndup

	and for the tools

	gnulib-tool --add-import --lib=libgnutools --source-base=tools/gnulib \
	--m4-base=tools/gnulib/m4 --aux-dir=build-aux --libtool \
	--macro-prefix=gl_tools --no-vc-files \
	getopt-gnu malloc-gnu progname version-etc

	More modules might have been added since. The currently-used gnulib
	modules and other gnulib information are recorded in
	~gnulib/m4/gnulib-cache.m4~ and ~tools/gnulib/m4/gnulib-cache.m4~.

	If you want to update from the current gnulib, install gnulib, and
	then run the following commands in the top-level directory.

	gnulib-tool --add-import --lib=libgnu --source-base=gnulib \
	--m4-base=gnulib/m4 --aux-dir=build-aux --libtool \
	--macro-prefix=gl --no-vc-files
	gnulib-tool --add-import --lib=libgnutools --source-base=tools/gnulib \
	--m4-base=tools/gnulib/m4 --aux-dir=build-aux --libtool \
	--macro-prefix=gl_tools --no-vc-files

	** How to build
	After getting the sources from git, with

	git clone https://github.com/liblouis/liblouis.git

	and installing the tools above, change to the liblouis directory and
	and bootstrap the project with the following command

	./autogen.sh

	to do a fresh build. Then run configure as usual:

	./configure

	You have the choice to compile liblouis for either 16- or 32-bit
	Unicode. By default it is compiled for the former. To get 32-bit
	Unicode run configure with --enable-ucs4 .

	After running configure run "make" and then "make install". You must
	have root privileges for the installation step.

	** Install with Homebrew
	Homebrew (http://brew.sh) is a package manager for Mac OS X that
	installs software from source. There is nothing special about the
	installation process in the sense that under the hood it happens
	exactly as described above, with the only difference that Homebrew
	automates it completely.

	First, use the ~brew tap~ command to add the repository that includes
	the liblouis formula:

	brew tap liblouis/liblouis

	Now you are ready to install liblouis:

	brew install liblouis

	** Docker
	Docker (https://www.docker.com) can be useful both for creating a
	development environment for liblouis, and for shipping the
	application.

	Setting up a developer environment can take long and can be
	problematic especially for Windows. Thanks to Docker we can set up the
	environment for you, we can easily distribute it as an image, which
	can be run by anybody and will behave exactly the same for everybody.

	Docker images of liblouis are being built automatically each time
	something changes in the code (see https://registry.hub.docker.com/repos/liblouis).
	In order to use them, first get Docker at
	http://docs.docker.com/introduction/get-docker. Download the latest
	liblouis image with:

	docker pull liblouis/liblouis

	Then, enter the development environment by running the image in a
	Docker container:

	docker run -it liblouis/liblouis bash

	Local files and directories can be "mounted" inside the container, in
	order to make it easier to edit files and to persist changes across
	runs. For example, to use local table files:

	docker run -it -v $(pwd)/tables:/tmp/liblouis/tables liblouis/liblouis bash

	See the Docker documentation for more info.

	The same Docker image can be used as a development environment and as
	the application itself. For example, to run the lou_translate tool
	from inside a Docker container:

	docker run -it liblouis/liblouis lou_translate en-us-g1.ctb

	To rebuild the image yourself, run the following command in the root
	directory of the liblouis source:

	docker build -t liblouis/liblouis .

	A .dockerignore file is required if you want to compile the source both on te host and in the Docker
	container. The .dockerignore file can be updated from .gitignore with:

	make .dockerignore

	** How to run tests
	Tests are run with

	make check

	** How to debug
	First you have to build liblouis with debugging info enabled.

	$ ./configure CFLAGS='-g -O0 -Wall -Wextra'
	$ make

	Starting the programs under the tools directory within gdb is a little
	tricky as they are linked with libtool. See the info page of libtool
	for more information. To start lou_checktable for table wiskunde.ctb
	for example you'd have to issue the following commands:

	$ libtool --mode=execute gdb ./tools/lou_checktable
	(gdb) run tables/wiskunde.ctb

	** How to find memory leaks
	Valgrind is a tool that can be used to find memory errors. It is
	recommended that you compile liblouis without any optimizations and
	with all warnings enabled before running it through Valgrind:

	$ ./configure CFLAGS='-g -O0 -Wall'
	$ make

	Then use Valgrind to analyze liblouis. For example you can run
	lou_translate trough Valgrind:

	$ libtool --mode=execute valgrind -v --tool=memcheck \
	--leak-check=full --leak-resolution=high --log-file=valgrind.log \
	./tools/lou_translate en-us-g2.ctb

	Type a few words at the prompt, check translation and terminate
	lou_translate. Now open the file valgrind.log and see if there are any
	memory leaks reported.

	You can also just run lou_checktable for example:

	$ libtool --mode=execute valgrind -v --tool=memcheck \
	--leak-check=full --leak-resolution=high --log-file=valgrind.log \
	./tools/lou_checktable tables/nl-BE-g0.utb

	Again open valgrind.log to see if any memory leaks were reported.

	For the full experience run lou_allround under Valgrind:

	$ libtool --mode=execute valgrind -v --tool=memcheck \
	--leak-check=full --show-reachable=yes \
	--leak-resolution=high --track-origins=yes \
	--log-file=valgrind.log ./tools/lou_allround

	** How to analyze performance
	Gprof helps you analyze the performance of programs. You have to
	compile liblouis as follows:

	$ ./configure --disable-shared
	$ make clean all CFLAGS='-g -O0 -pg' LDFLAGS='-all-static'

	Then translate some stuff with a large table:

	$ ./tools/lou_translate tests/tables/large.ctb

	Finally look at the call profile:

	$ libtool --mode=execute gprof ./tools/lou_translate gmon.out

	** How to build for Windows
	See the README.windows file and the windows subdirectory.

	*** How to cross-compile for Windows
	To compile for win32, use the MinGW win32 cross-compiler as shown
	below. Use the prefix option to install the binaries to a temporary
	place where you can create a zip file. The LDFLAGS='-all-static'
	ensures that libgcc is linked in statically. Otherwise the users need
	to have libgcc_s_sjlj-1.dll.

	Some users want the dlls unversioned. To achieve that add
	'-avoid-version' to LDFLAGS.

	./configure --build=i686-pc-linux-gnu --host=i686-w64-mingw32 --prefix=/tmp/liblouis-mingw32
	make LDFLAGS='-avoid-version -Xcompiler -static-libgcc'
	make install
	zip -r liblouis-mingw32msvc.zip /tmp/liblouis-mingw32

	To compile for win64, use the MinGW-w64 cross-compiler:

	./configure --build=i686-pc-linux-gnu --host=x86_64-w64-mingw32 --prefix=/tmp/liblouis-w64-mingw32
	make LDFLAGS='-avoid-version -Xcompiler -static-libgcc'
	make install
	zip -r liblouis-w64-mingw32.zip /tmp/liblouis-w64-mingw32

	Two makefile rules have been provided to do this automatically for you. Docker will be used if the
	right cross-compiler is not installed. To compile for win32 using the MinGW cross-compiler, run make
	distwin32. This will produce a ZIP file called liblouis-<version>-win32.zip. To compile for win64
	using the MinGW-w64 cross-compiler, run make distwin64. This will produce a ZIP file called
	liblouis-<version>-win64.zip.

	*** TODO How to build for Windows using Cygwin
	(possibly use a Vagrantfile as demonstration + explain that Cygwin
	binaries can not be used outside the Cygwin environment)

	*** TODO How to build for Windows using MinGW
	(possibly use a Vagrant file as demonstration)


	* Release Procedure
	These steps describe what a maintainer does to make a release; they
	are not needed for ordinary patch submission.

	** Update the NEWS entry
	If any new tables were added or renamed please note them with their
	name in the NEWS entry. This is usefull for projects like NVDA.

	** Set the version number
	Update the version number in NEWS (with version, date, and release
	type), ChangeLog, configure.ac and windows/include/config.h.

	Don't forget to update the libtool versioning info in configure.ac,
	i.e. LIBLOUIS_REVISION and possibly LIBLOUIS_CURRENT and LIBLOUIS_AGE.

	** Commit and tag
	Commit the changes and tag this version

	git tag -s v2.6.0 -m "Release 2.6.0"
	git push origin v2.6.0

	If you know the exact version number that needs to be tagged use

	git tag -s v2.6.0 -m "Release 2.6.0" <commit>
	git push origin v2.6.0

	** Make the release
	Check out a clean copy in a different directory, like /tmp. Run
	autogen.sh and configure with no special prefixes. Run make distcheck.
	This will make sure that all needed files are present, and do a
	general sanity check. Run make dist. This will produce a tarball.

	./autogen.sh && ./configure && make && make distcheck && make dist

	** Upload
	Add the tarball to the github liblouis releases page, i.e. add it
	under https://github.com/liblouis/liblouis/releases with the specific
	release and add a link to it in $WEBSITE/downloads/index.md. See below
	for instructions on how to update the web site.

	** Online documentation
	The online documentation is part of the liblouis web site. To add it to the
	site simply copy doc/liblouis.html to $WEBSITE/documentation/liblouis.html.
	Make sure you add the proper YAML front matter. Again see below for
	instructions on how to update the web site.

	** Web site maintenance
	The liblouis web site at liblouis.org is maintained with the help of
	github pages (https://pages.github.com/). To edit the site just check
	out the repo at https://github.com/liblouis/liblouis.github.io. You'll
	need to know a few things about Jekyll (http://jekyllrb.com/) and
	markdown, the markup that is used to edit the content. In order to
	update the site simply edit, commit and push.

	For the new release update the project web site.

	- Add a post containing the current NEWS to the _posts directory and
	- add the download artifacts to the download page

	** Announce
	Send an announcement to the liblouis list
	liblouis-liblouisxml@freelists.org. See ANNOUNCEMENT for an example.