Features Download

From: Charles Wilson <cwilso11-Rn4VEauK+AKRv+LV9MX5uipxlwaOVQ5f <at> public.gmane.org>
Subject: Re: New packages
Newsgroups: gmane.comp.gnu.mingw.devel
Date: Tuesday 23rd June 2009 00:25:19 UTC (over 9 years ago)
Keith Marshall wrote:
> On Sunday 21 June 2009 23:38:19 Charles Wilson wrote:
>> I propose that the autotools
>> in category #1 follow this naming convention:
>> autoconf-6-1-mingw32-bin.tar.lzma  (wrapper)
> Looks good to me.

Except that this contradicts your comments below, concerning -mingw- vs.
-msys- vs. -msysDVLPR-

>> I'm also open to (a) possibly splitting the documentation out into
>> a -doc pkg for each,
> Makes sense, if it's easily separated out.

Easy peasy.

>> and/or (b) using -dev in preference to -bin 
>> for the package that contains the actual scripts.
> I'd be inclined to stick with -bin; -dev really implies header files, 
> libraries, (and maybe associated docs, if not separated).  Autoconf 
> doesn't provide such components, so a -dev package here would seem 
> to be an anomaly.

That was my thought as well.

>> The real controversy here is the use of "mingw32" as the subsystem
>> identifier.  Technically, these packages are *msys* utilities,
>> because they depend on msys-perl, msys-bash, etc.
> Yes, and to classify them as such really does seem more natural, than 
> to suggest that they may be MinGW utilities, which most would expect 
> to be able to run independently of MSYS.
>> However, the 
>> question is how to distinguish THESE packages from the msysdvlpr
>> ones -- I don't really like "-msys-"  vs.  "-msysdvlpr-".
> I take your point, but I am concerned that autoconf-*-mingw32 may 
> lead to end user confusion, and an expectation that these should run 
> without MSYS, where in fact this isn't the case.  If -MSYS- alone is 
> to serve for tools used in MSYS with the MINGW32 host personality -- 
> as is the case for the majority of other packages -- then we really 
> do need a distinct identifier for MSYSDVLPR.  Whether that ends up 
> as -MSYSDVLPR-, -MSYS-DVLPR-, -MSYS-SDK- or some other moniker -- 
> I'm open to suggestions -- it really *shouldn't* be just -MSYS-.

So here's the real kicker: libtool.  From your reasoning, then, we
should have:

libtool-2.2.7a-1-mingw32-dll-7.tar.lzma (contains a mingw DLL)
libtool-2.2.7a-1-msys-bin.tar.lzma      (contains an (msys)bash script)
libtool-2.2.7a-1-???-doc.tar.lzma       (ok, now I'm confused)
libtool-2.2.7a-1-???-src.tar.lzma       (and this makes my head hurt)

I'd guess there would be a lot of packages like that. Take your/my
native build of bzip2. The .exe's, dll's, and libs are all -mingw-. BUT,
bzip2 also provides
which are all (msys)sh scripts.  So, my -bin package which contains
these scripts, plus the perfectly usable native bzip2.exe, is what?


I think we're going to have to accept the following:
  1) sometimes there is no "correct" choice -- all choices are to
     some degree wrong.
  2) we should err on the side of least surprise (or, when all choices
are to some extent surprising, fall back to the hobgoblin of little
minds like my own: consistency)

So, for my bzip2 example, I'd definitely call it -mingw32-, and perhaps
add a blurb to the ReleaseNotes that "executables and libraries provided
by this package are native win32, but some of the scripts provided
require msys to operate correctly" and let it go.

For libtool...what's more important: libltdl-7.dll is mingw32, or
libtool requires (a lot) of msys support?  I could go either way...punt
to "whatever we decide for the rest of the autotools".  Principle of
least surprise: all the autotool packages for MSYS-as-MinGW-personal
should at least use the SAME subsys id.

Concerning gettext, I do consider to be an autotool because of
autopoint.  libintl-2.dll is mingw.  All the .exe utilities are mingw.
But autopoint is an (msys)sh script, and requires (msys)cvs.  MOST of
gettext is mingw32.

Similarly, (the native build of) gettext's apps and dll's depend on (the
native build of) libiconv -- which as currently distributed contains
nothing which relies on msys.  So certainly libiconv should be -mingw32-
(-msys- would be just silly, AND misleading).

But, back to "use the SAME subsys id" -- by my argument, libiconv and
gettext should be -mingw32-.  BUT gettext::autopoint requires msys
support programs.

Now, that just knocks the legs out of the argument that the libtool
"script" forces all of the libtool packages to use the -msys- subsys id.
A tie (libtool-bin vs. libtool-dll) goes to the runner (gettext &
libiconv use -mingw32-, and gettext-bin contains autopoint, just like
libtool-bin contains libtool proper).

Which finally brings us back to automake/autoconf.  Well, if
libtool/gettext/libiconv all claim -mingw32-, these boys should too.

>> And 
>> besides, these really should reside in /mingw/bin, so that the
>> typical PATH settings which have /mingw/bin precede /usr/bin when
>> !msysdvlpr will enable the typical mingw user to have default
>> access to the "mingw32"-specific autotool utilities, and not
>> "accidentally" get the msys-specific ones.
> Agreed.  However, I've never really understood why the one set of 
> autotools can't suffice for both purposes, (with the exception of 
> libtool perhaps, and gettext which I don't view as a natural 
> component of the autotools in any case).

While autoconf and automake are platform-agnostic [*], libtool is not,
as you point out.  But it's worse: the autotools expect to work
together, in that aclocal creates aclocal.m4 using m4 fragments from
  1) local m4/ dir
  2) $prefix/share/aclocal
  3) $prefix/ahare/aclocal-X.Y
  4) wherever $prefix/share/aclocal/.dirlist says
AND the aclocal.m4 it creates is supposed to work with the
autoconf/automake you use.  Ditto libtoolize is going to copy ITS
$ltprefix/share/aclocal/libtool.m4 & friends into the local m4/ dir for
aclocal to use -- and that better match the ltmain.sh you have, as well.

[*] contrary to early reports from long ago, autoconf and automake
should NOT be configured with the typical mingw '--prefix=`cd /mingw &&
pwd -W`'.  This is because they are entirely comprised of perl and shell
scripts, and in our environment rely on (msys)perl and (msys)bash. Thus,
throwing C:\Dos\Style\Path\With Spaces\ at them just doesn't work.  They
actually want --prefix=/mingw (or $unix_prefix_of_your_choice).

Then, consider all the libraries that stick their .m4 files (esp.
gettext! Wow, it supplies almost as many .m4 files as automake itself)

Now, suppose I have only ONE common installation of autoconf and
automake, to be used by both msysdvlpr and msys-as-mingw environments.
Suppose this is the "msys" build of those tools, and thus was configured
with --prefix=/usr NOT --prefix=/mingw.  There is only one search path
for .m4 files -- /usr/share/aclocal/ and friends.

Well, I've got a native build of libxml configured and installed, like a
good little mingw user, in /mingw.  I'm developing a package that uses
that library, and I want to reference libxml.m4 from my package's
configure.ac.  /usr/bin/aclocal will look in /usr/share/aclocal/ and
friends, and will NOT find libxml.m4 unless
  a) I explicitly use 'aclocal -I /mingw/share/aclocal'
  b) I add ACLOCAL_FLAGS = -I /mingw/share/aclocal to my project's
     Makefile.am.  That's not very cross-platform...
  c) I add /mingw/share/aclocal to /usr/share/aclocal/.dirlist
     However, this has other problems, in that it breaks the msysdvlpr
usage...there's now an .m4 file in aclocal's search path associated with
a library for which I don't have an msys build (so what? you say. Well,
you're right: this isn't REALLY a problem. It's not presence/vs/absence
that is the real trouble. It's version mismatch. See below)

None of these choices seem good to me.  It oughta Just Work without
extra effort on the developer's part -- or platform specific
ACLOCAL_FLAGS settings in a supposedly cross-platform Makefile.am.

So here's the REAL problem with c): different VERSIONS of an .m4 file.
We *will* have multiple versions -- native and msys -- of libiconv and
libintl/gettext.  Each provides lots of .m4 files.  These .m4 files are
not always intercompatible across versions.

So, to allow a single autoconf installation, I/We have to ensure that
libiconv-win32 and libiconv-msys are always kept in sync, so that the
.m4 files stay compatible. Ditto libintl-win32 and libintl-msys.  AND we
have to ensure that all USERS always upgrade both simultaneously if they
have them installed at all.

IMO it's easier to simply have a separate entire autotoolchain for msys,
and a separate entire autotoolchain for native.  What 'subsys id' we
give em', I dunno.  See above.

Plus, this actually simplifies things a bit. I can distribute just One
True Autoconf Version and One True Automake Version for msysdvlpr (plus,
recall that the msys versions are patched to support the -msys- $host
triple, which Earnie explicitly does NOT want leaking outside of the
msysdvlpr environment).

The mingw autotoolchains can be all tricked out with multiple versions
and wrapper scripts and whatnot -- and I don't have to worry if all that
junk actually WORKS in the msysdvlpr environment.  I just have to ensure
that the One True Version works in msysdvlpr.

>> (Another aside: I plan to eventually create binary mingw packages
>> for:
>>    zlib 
>>    bzip2
>>    xz
> I was planning to do these myself; (already have them built, and just 
> the split, packaging and distribution to complete).  Do you want to 
> take them over, or shall I continue with them?

It's probably better if I take over xz just for synergy. I'm already
bouncing patches back and forth with Lasse Collin concerning the build

I also maintain the cygwin builds of zlib, mingw-zlib [*], bzip2, and
mingw-bzip2 [*], so I could do those two as well, but in the interests
of preserving my sanity, how about you go ahead with zlib and bzip2?

[*] these are actually mingw-native builds, but are built using cygwin
gcc -mno-cygwin, are intended SOLELY to support developers of setup.exe,
and whose DLL names are munged to avoid conflict with anything the MinGW
project might ever distribute. There will soon also be a cygwin mingw-xz

>>    libarchive
> However, I ran into a brick wall with this -- and please *don't* tell 
> me I need to kludge it, by using CMake to generate a build system.  
> If I can't build it with a simple
>   ../path/to/configure --build=i686-pc-linux --host=mingw32 ...
> I consider it to be broken.  

I agree. I'm actually one of the project members for libarchive. I've
helped to get it building on cygwin (configure/make) but haven't had a
chance to do the same for mingw.  That's already on my list...

> (FTR, libarchive-2.7.0 appears to 
> configure fine, but the source then fails to compile correctly; it 
> breaks at the outset, in libarchive/archive-entry.c, which is the 
> very first file in the compilation sequence)[*].

Cute.  That always gives me warm fuzzies.

>> specifically so that we can distribute/have available a
>> statically-linked native win32 bsdtar.exe that supports lzma, gz,
>> and bzip2 without delegating to external decompression,
> Yep.  That was my motivation for playing with zlib, libbz2, liblzma 
> and libarchive, but I wasn't aiming for a standalone bsdtar -- I'd 
> like to integrate the unpacking function directly into mingw-get.

Well, once the libraries are available, all sorts of things become

> I started out with the groundwork John E. has already put in place,
> (but eschewing his decomposition of the prerequisite libraries in 
> favour of linking against the full library code from upstream).

That's my preferred approach, too.

> I 
> had no difficulty in getting the decompression filters working -- 
> gzip from libz, bzip2 from libbz2, lzma and xz from liblzma.  Rather 
> than hack around the issues with libarchive, I took Tim Kientzle's 
> untar.c, and adapted it.

That'll work.

>> which IIRC GNU tar doesn't do well on native win32.
> Maybe.  I don't know.  I've never attempted to build GNU tar for 
> native w32.  My understanding is that bsdtar is better in any case, 
> and I was disappointed that libarchive failed to build.

GNU tar tries to fork/exec external decompression programs except for
compress, gz, and bz2.  Without lots of TLC (or gnulib code) this
doesn't work on native win32 the way GNU tar expects it to.

> [*] Yes, I am aware that there are known issues with 2.7.0 and MinGW;
> the issues I saw -- very quickly -- didn't seem to be related to the 
> issues I saw reported on the libarchive wiki.

Well, it oughta work. And it WILL work. Just as soon as I can get to it.


Are you an open source citizen? Join us for the Open Source Bridge
Portland, OR, June 17-19. Two days of sessions, one day of unconference:
Need another reason to go? 24-hour hacker lounge. Register today!
CD: 163ms