pkgsrc on SmartOS - creating new packages
Having set up a build zone and started fixing packages, the next area we will look at is creating new packages. For this tutorial I will use augeas as an example, as it happens to be a package I recently created.
First, install the
pkgtools/url2pkg package, as this massively simplifies the task in hand.
: If you have pkgin and binary packages available.. $ sudo pkgin in url2pkg : ..else build from source $ (cd pkgtools/url2pkg; bmake install)
Then create a new package directory, choosing the most appropriate category (in this case
devel), then run
url2pkg giving it an argument of the source tarball.
$ mkdir devel/augeas $ cd devel/augeas $ url2pkg http://download.augeas.net/augeas-1.0.0.tar.gz
First it will open an editor session on
Makefile. You should customise a few variables:
MAINTAINEReither to your email address, or to
pkgsrc-users@NetBSD.orgif you do not want to be the primary maintainer of the package.
Write a brief one-line
COMMENTdescribing what the package is.
LICENSEto a list of the licenses the package is made available under (see
mk/license.mkfor an available list).
Here is what I wrote for augeas:
# $NetBSD$ DISTNAME= augeas-1.0.0 CATEGORIES= devel MASTER_SITES= http://download.augeas.net/ MAINTAINER= pkgsrc-users@NetBSD.org HOMEPAGE= http://download.augeas.net/ COMMENT= Configuration file editing tool and library LICENSE= gnu-lgpl-v2 # url2pkg-marker (please do not remove this line.) .include "../../mk/bsd.pkg.mk"
After writing the file and exiting the editor session,
url2pkg will continue and download the source tarball, create the
distinfo files, then unpack the source ready for you to start applying patches.
At this point you should edit
DESCR and put in a few lines which describe the package.
The general cycle will then be:
Try building the package with
If something goes wrong, modify the
Makefileor patch the source with
until the package builds. Before running your first
bmake, I would strongly recommend adding
PKG_DEVELOPER=yes to your
mk.conf to turn on a lot of additional checks.
augeas, I needed a couple of things.
The first problem I hit was:
$ bmake [...] => Checking for portability problems in extracted files ERROR: [check-portability.awk] => Found test ... == ...: ERROR: [check-portability.awk] configure: test "`uname`" == "SunOS" && \ Explanation: =========================================================================== The "test" command, as well as the "[" command, are not required to know the "==" operator. Only a few implementations like bash and some versions of ksh support it. When you run "test foo == foo" on a platform that does not support the "==" operator, the result will be "false" instead of "true". This can lead to unexpected behavior. There are two ways to fix this error message. If the file that contains the "test ==" is needed for building the package, you should create a patch for it, replacing the "==" operator with "=". If the file is not needed, add its name to the CHECK_PORTABILITY_SKIP variable in the package Makefile. ===========================================================================
In this case pkgsrc gives us a very helpful message telling us about the problem and what needs to be done to fix it (unfortunately not all problems are handled this well!). As
configure is pretty important, we’ll need to patch it.
This is my usual way of handling patches (ensuring
pkgtools/pkgdiff is installed first):
: Set up patches directory and some useful variables. $ mkdir patches $ patchdir=$(pwd -P)/patches $ wrkdir=$(bmake show-var VARNAME=WRKSRC) : Edit file and generate patch $ cd $wrkdir $ pkgvi configure $ pkgdiff configure >$patchdir/patch-configure : Return to pkgsrc directory, re-generate distinfo and try again. $ cd - $ bmake clean; bmake mps; bmake
This was enough to fix the
== problem. I then ran into other issues.
Dependencies in pkgsrc are primarily handled in two ways, either with a
buildlink file when depending upon shared libraries or particular infrastructure, or a simple
DEPENDS line to pull in a required package.
The first dependency problem I hit was:
[...] checking readline/readline.h usability... no checking readline/readline.h presence... no checking for readline/readline.h... no checking for readline in -lreadline... no checking for library containing tgetent... -lcurses checking for rl_initialize in -lreadline... no configure: WARNING: readline library not found configure: error: Could not find a working readline library (see config.log for details). *** Error code 1
Ok, so this package depends upon readline. As that is a library, we are looking for a suitable
buildlink3.mk file, and one of the easiest ways to do this is:
$ ls -1d ../../*/*readline*/buildlink3.mk ../../devel/readline/buildlink3.mk
To use that file, all we need to do is add an
.include line at the bottom of
Makefile, just above the
.include "../../devel/readline/buildlink3.mk" .include "../../mk/bsd.pkg.mk"
bmake clean; bmake to try again. I then hit:
[...] checking for LIBXML... no configure: error: Package requirements (libxml-2.0) were not met: No package 'libxml-2.0' found
Ok, so similar method:
$ ls -1d ../../*/*libxml*/buildlink3.mk ../../textproc/libxml++/buildlink3.mk ../../textproc/libxml++1/buildlink3.mk ../../textproc/libxml/buildlink3.mk ../../textproc/libxml2/buildlink3.mk ../../textproc/py-libxml2/buildlink3.mk
configure message said libxml-2.0, so we’ll pick
textproc/libxml2. If you want to be more thorough you can look at the
PLIST files for each of the possible candidate packages - I just happen to know from experience that
textproc/libxml2 is the one I need.
The bottom of
Makefile now looks like this:
.include "../../devel/readline/buildlink3.mk" .include "../../textproc/libxml2/buildlink3.mk" .include "../../mk/bsd.pkg.mk"
You’ll note that we try to keep all includes other than
bsd.pkg.mk (which is special) sorted alphabetically.
After this, the package finally completes a
bmake with no problems. The final step is to get the
install phase working.
First we need to run a
stage-install which will execute
make install into a temporary
$ bmake stage-install
This will almost certainly fail as we haven’t configured the
PLIST yet, so pkgsrc has no idea what will be installed from this package.
[...] => Automatic manual page handling => Generating post-install file lists => Checking file-check results for augeas-1.0.0 ERROR: ************************************************************ ERROR: The following files are in /var/tmp/pkgsrc-build/devel/augeas/work/.destdir/opt/pkg but not in the PLIST: ERROR: /var/tmp/pkgsrc-build/devel/augeas/work/.destdir/opt/pkg/bin/augparse ERROR: /var/tmp/pkgsrc-build/devel/augeas/work/.destdir/opt/pkg/bin/augtool [...]
However, now that we have a populated
DESTDIR, we can use the
print-PLIST target to generate it for us:
$ bmake print-PLIST >PLIST
Finally, it’s worth doing a full clean and install to ensure everything works as expected.
$ bmake clean $ bmake install [...] => Creating binary package /var/tmp/pkgsrc-build/devel/augeas/work/.packages/augeas-1.0.0.tgz ===> Install binary package of augeas-1.0.0
As we are installing a library package, we should provide a
buildlink3.mk file of our own so that other packages can depend on us correctly. Again, there is a package that can help with this -
$ createbuildlink >buildlink3.mk
This will create a template file but you should read and edit, removing the lines marked with
XXX and making any changes they recommend.
Once we have a working package it’s worth doing a couple of checks to make sure everything looks ok:
: Check the package listing looks ok $ pkg_info -L augeas : View the included DESCR and homepage $ pkg_info -d augeas : Check what the package lists as its dependencies $ pkg_info -N augeas
We also should run
pkgtools/pkglint) which will perform some sanity checks on our infrastructure files prior to us importing the package into pkgsrc itself. In my case it pointed out some possible issues:
$ pkglint WARN: PLIST:8: Packages that install libtool libraries should define USE_LIBTOOL. WARN: PLIST:9: Packages that install libtool libraries should define USE_LIBTOOL. ERROR: patches/patch-configure:3: Comment expected. 1 errors and 2 warnings found. (Use -e for more details.)
For the first issue, add
Makefile which will ensure that the pkgsrc libtool is used (has better support for cross-platform issues), and for the second, add a comment at the top of the file explaining what it is for, then re-generate
After making those changes, everything looks great:
$ pkglint looks fine.
The final - and sometimes most challenging - part is to get your shiny new package integrated into pkgsrc. There are a few options:
Find a NetBSD developer (like myself) who can review your package and put it straight into pkgsrc.
Get a pkgsrc-wip account and work on your packages there, before getting it reviewed and integrated.
If it’s somewhat niche just publish it up on GitHub or similar for people to use, in a similar way to what we do with pkgsrc-joyent
This post only scratches the surface of adding a new package. pkgsrc provides a huge amount of infrastructure to help get packages working on multiple platforms, and there are lots of options available.
Certain parts of the infrastructure, like buildlink, are very complicated, and for more in-depth information you should refer to the pkgsrc guide.
The best way is often just to look at other packages to see what they do and re-use useful bits you find. With 12,000 packages there is almost certainly another package which does something similar to what you need!
- 06 Oct 2014 » Building packages at scale
- 04 Dec 2013 » A node.js-powered 8-bit CPU - part four
- 03 Dec 2013 » A node.js-powered 8-bit CPU - part three
- 02 Dec 2013 » A node.js-powered 8-bit CPU - part two
- 01 Dec 2013 » A node.js-powered 8-bit CPU - part one
- 21 Nov 2013 » MDB support for Go
- 30 Jul 2013 » What's new in pkgsrc-2013Q2
- 24 Jul 2013 » Distributed chrooted pkgsrc bulk builds
- 07 Jun 2013 » pkgsrc on SmartOS - creating new packages
- 15 Apr 2013 » What's new in pkgsrc-2013Q1
- 19 Mar 2013 » Installing SVR4 packages on SmartOS
- 27 Feb 2013 » SmartOS is Not GNU/Linux
- 18 Feb 2013 » SmartOS development preview dataset
- 17 Jan 2013 » pkgsrc on SmartOS - fixing broken builds
- 15 Jan 2013 » pkgsrc on SmartOS - zone creation and basic builds
- 10 Jan 2013 » Multi-architecture package support in SmartOS
- 09 Jan 2013 » Solaris portability - cfmakeraw()
- 08 Jan 2013 » Solaris portability - flock()
- 06 Jan 2013 » pkgsrc-2012Q4 illumos packages now available
- 23 Nov 2012 » SmartOS and the global zone
- 24 Oct 2012 » Setting up Samba on SmartOS
- 10 Oct 2012 » pkgsrc-2012Q3 packages for illumos
- 23 Aug 2012 » Creating local SmartOS packages
- 10 Jul 2012 » 7,000 binary packages for OSX Lion
- 09 Jul 2012 » 9,000 packages for SmartOS and illumos
- 07 May 2012 » Goodbye Oracle, Hello Joyent!
- 13 Apr 2012 » SmartOS global zone tweaks
- 12 Apr 2012 » Automated VirtualBox SmartOS installs
- 30 Mar 2012 » iptables script for Debian / Ubuntu
- 20 Feb 2012 » New site design
- 11 Jan 2012 » Set up anonymous FTP upload on Oracle Linux
- 09 Jan 2012 » Kickstart Oracle Linux in VirtualBox
- 09 Jan 2012 » Kickstart Oracle Linux from Ubuntu
- 22 Dec 2011 » Last day at MySQL
- 15 Dec 2011 » Installing OpenBSD with softraid
- 21 Sep 2011 » Create VirtualBox VM from the command line
- 14 Sep 2011 » Creating chroots for fun and MySQL testing
- 30 Jun 2011 » Graphing memory usage during an MTR run
- 29 Jun 2011 » Fix input box keybindings in Firefox
- 24 Jun 2011 » How to lose weight
- 23 Jun 2011 » How to fix stdio buffering
- 13 Jun 2011 » Serving multiple DNS search domains in IOS DHCP
- 13 Jun 2011 » Fix Firefox URL double click behaviour
- 20 Apr 2011 » SSH via HTTP proxy in OSX
- 09 Nov 2010 » How to build MySQL releases
- 29 Apr 2010 » 'apt-get' and 5,000 packages for Solaris10/x86
- 16 Sep 2009 » ZFS and NFS vs OSX
- 12 Sep 2009 » pkgsrc on Solaris
- 09 Dec 2008 » Jumpstart from OSX
- 31 Dec 2007 » Set up local caching DNS server on OSX 10.4