== PostgreSQL Weekly News - September 02 2012 ==
Philly PUG will have its first meeting Sept 12, 2012. Bruce Momjian
presents: What's new in PostgreSQL 9.2. Details and R, SVP, below.
PgDay Argentina 2012, a mini conference focused on connecting bussines
& new users with the PostgreSQL community, will take place November
13th, 2012 in Buenos Aires, Argentina.
This year Ecuador's PGDay will be held at Quito city on November 17th,
2012. The Call for Papers is open until October 10. Send proposals
to Jaime Casanova jaime 2ndquadrant com.
The sixth edition of the Italian PostgreSQL Day (PGDay.IT 2012) will
be held on November 23, 2012 in Prato, Tuscany. The International
Call for Papers is now open. Talks and presentations in English are
accepted. Information in English for papers submission is available
== PostgreSQL Local ==
PostgreSQL Session will be held on October 4th, 2012, in Paris,
France. More information at:
PostgreSQL Conference Europe 2012 will be in Prague, Czech Republic
on October 23-26.
PostgreSQL Day Argentina 2012 will be held on November 13th in Bernal,
Buenos Aires, at the National University of Quilmes. It will cover
topics for PostgreSQL users, developers and contributors, as well as
decision and policy makers. For more information about the
conference, please see the website at
== PostgreSQL in the News ==
Planet PostgreSQL: http://planet.postgresql.org/
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to email@example.com, German language
to firstname.lastname@example.org, Italian language to email@example.com. Spanish language
== Applied Patches ==
Heikki Linnakangas pushed:
- Collect and use histograms of lower and upper bounds for range
types. This enables selectivity estimation of the <<, >>, &<, &>
and && operators, as well as the normal inequality operators: <, <=,
>=, >. "range @> element" is also supported, but the range-variant
@> and <@ operators are not, because they cannot be sensibly
estimated with lower and upper bound histograms alone. We would need
to make some assumption about the lengths of the ranges for that.
Alexander's patch included a separate histogram of lengths for that,
but I left that out of the patch for simplicity. Hopefully that
will be added as a followup patch. The fraction of empty ranges is
also calculated and used in estimation. Alexander Korotkov, heavily
modified by me.
- Optimize SP-GiST insertions. This includes two micro-optimizations
to the tight inner loop in descending the SP-GiST tree: 1. avoid an
extra function call to index_getprocinfo when calling user-defined
choose function, and 2. avoid a useless palloc+pfree when node
labels are not used.
- Fix division by zero in the new range type histogram creation.
Report and analysis by Matthias.
- Fix typos in README.
Bruce Momjian pushed:
- Have pgindent requre pg_bsd_indent version 1.2 now that a new
version has been created by adding #include to parse.c.
Per request from Kevin Grittner.
- Document the lack of reading the PGDATABASE environment variable in
- Document how to create advisory lock "bigint" values in SQL. David
- Update doc mention of how to compute bigint advisory lock value, per
Tom Lane's correction.
- Prevent psql tab completion in SET from adding TO when the equals
sign has no space before it. Report by Erik Rijkers
- Adjust pg_test_timing to show shortest test durations first, place
percentage column before count column. Docs updated.
- Report postmaster.pid file as empty if it is empty, rather than
reporting in contains invalid data.
- Document that NOTIFY events are visible to all users.
- Remove old documentation warnings about the use of bigint.
- Document that log_autovacuum_min_duration can be used to monitor
autovacuum activity. Per report from Marc Mamin
- Remove configure flag --disable-shared, as it is no longer used by
any port. The last use was QNX, per Peter Eisentraut.
- Document that max_locks_per_transaction might need to be increased
for queries on parent tables, per suggestion from Josh Berkus.
- Remove doc mention of --enable-shared, per Tom.
- Revert 1fbc30f1dca6624c01a9a6a9ff711a7036c68532 because the
--enable-shared is about Plyton's configure, not ours.
- Properly document that SIGTERM is OK for users to use on a postgres
session, now that pg_terminate_backend() uses it. Josh Kupershmidt
- Clarify FreeBSD instructions in pg_test_timing. Josh Kupershmidt
- Add Perl emacs formatting macros to src/tools/editors/emacs.samples.
- In pg_upgade, document that the port numbers must be different when
checking an old running server.
- Revert doc patch 305557984dd964ac397c6752e9d0f14646b60f15 as the
values are sometimes signed, sometimes unsigned.
- Add small doc mention that libpq is named after POSTQUEL.
- Update URLs that pointed to sun.com; either repoint them or remove
Tom Lane pushed:
- Fix DROP INDEX CONCURRENTLY IF EXISTS. This threw ERROR, not the
expected NOTICE, if the index didn't exist. The bug was actually
visible in not-as-expected regression test output, so somebody
wasn't paying too close attention in commit
8cb53654dbdb4c386369eb988062d0bbb6de725e. Per report from Brendan
- Add section IDs to subsections of syntax.sgml that lacked them.
This is so that these sections will have stable HTML tags that one
can link to, rather than things like "AEN1902". Perhaps we should
mount a campaign to do this everywhere, but I've found myself
pointing at syntax.sgml subsections often enough to be sure it's
- Improve a couple of 9.2 release note entries. Clarify the
compatibility notes about EXTRACT() and about statistics timing
- Remove hstore--1.0.sql. Since we're not installing this file
anymore, it has no reason to exist, other than as historical
reference; but we have an SCM for that.
- Remove pg_stat_statements--1.0.sql, too. Per Peter Geoghegan.
- Adjust definition of cheapest_total_path to work better with
LATERAL. In the initial cut at LATERAL, I kept the rule that
cheapest_total_path was always unparameterized, which meant it had
to be NULL if the relation has no unparameterized paths. It turns
out to work much more nicely if we always have *some* path nominated
as cheapest-total for each relation. In particular, let's still say
it's the cheapest unparameterized path if there is one; if not, take
the cheapest-total-cost path among those of the minimum available
parameterization. (The first rule is actually a special case of the
second.) This allows reversion of some temporary lobotomizations I'd
put in place. In particular, the planner can now consider hash and
merge joins for joins below a parameter-supplying nestloop, even if
there aren't any unparameterized paths available. This should bring
planning of LATERAL-containing queries to the same level as queries
not using that feature. Along the way, simplify management of
parameterized paths in add_path() and friends. In the original
coding for parameterized paths in 9.2, I tried to minimize the logic
changes in add_path(), so it just treated parameterization as yet
another dimension of comparison for paths. We later made it ignore
pathkeys (sort ordering) of parameterized paths, on the grounds that
ordering isn't a useful property for the path on the inside of a
nestloop, so we might as well get rid of useless parameterized paths
as quickly as possible. But we didn't take that reasoning as far as
we should have. Startup cost isn't a useful property inside a
nestloop either, so add_path() ought to discount startup cost of
parameterized paths as well. Having done that, the secondary
sorting I'd implemented (in add_parameterized_path) is no longer
needed --- any parameterized path that survives add_path() at all is
worth considering at higher levels. So this should be a bit faster
as well as simpler.
- Improve EXPLAIN's ability to cope with LATERAL references in plans.
push_child_plan/pop_child_plan didn't bother to adjust the
"ancestors" list of parent plan nodes when descending to a child
plan node. I think this was okay when it was written, but it's not
okay in the presence of LATERAL references, since a subplan node
could easily be returning a LATERAL value back up to the same
nestloop node that provides the value. Per changed regression test
results, the omission led to failure to interpret Param nodes that
have perfectly good interpretations.
- Suppress creation of backwardly-indexed paths for LATERAL join
clauses. Given a query such as SELECT * FROM foo JOIN LATERAL
(SELECT foo.var1) ss(x) ON ss.x = foo.var2 the existence of the join
clause "ss.x = foo.var2" encourages indxpath.c to build a
parameterized path for foo using any index available for foo.var2.
This is completely useless activity, though, since foo has got to be
on the outside not the inside of any nestloop join with ss. It's
reasonably inexpensive to add tests that prevent creation of such
paths, so let's do that.
- Improve coding of gistchoose and gistRelocateBuildBuffersOnSplit.
This is mostly cosmetic, but it does eliminate a speculative
portability issue. The previous coding ignored the fact that
sum_grow could easily overflow (in fact, it could be summing
multiple IEEE float infinities). On a platform where that didn't
guarantee to produce a positive result, the code would misbehave.
In any case, it was less than readable.
- Make configure probe for mbstowcs_l as well as wcstombs_l. We
previously supposed that any given platform would supply both or
neither of these functions, so that one configure test would be
sufficient. It now appears that at least on AIX this is not the
case ... which is likely an AIX bug, but nonetheless we need to cope
with it. So use separate tests. Per bug #6758; thanks to Andrew
Hastie for doing the followup testing needed to confirm what was
happening. Backpatch to 9.1, where we began using these functions.
- Fix LATERAL references to join alias variables. I had thought this
case worked already, but perhaps I didn't re-test it after adding
- Partially restore qual scope checks in distribute_qual_to_rels().
The LATERAL implementation is now basically complete, and I still
don't see a cost-effective way to make an exact qual scope
cross-check in the presence of LATERAL. However, I did add a
PlannerInfo.hasLateralRTEs flag along the way, so it's easy to make
the check only when not hasLateralRTEs. That seems to still be
useful, and it beats having no check at all.
- More documentation updates for LATERAL. Extend xfunc.sgml's
discussion of set-returning functions to show an example of using
LATERAL, and recommend that over putting SRFs in the targetlist. In
passing, reword func.sgml's section on set-returning functions so
that it doesn't claim that the functions listed therein are all the
built-in set-returning functions. That hasn't been true for a long
time, and trying to make it so doesn't seem like it would be an
improvement. (Perhaps we should rename that section?) Both per
suggestions from Merlin Moncure.
- Fix broken link in installation.sgml. Linking to other parts of the
manual doesn't work when building the standalone INSTALL document.
- Fix mark_placeholder_maybe_needed to handle LATERAL references. If
a PlaceHolderVar contains a pulled-up LATERAL reference, its minimum
possible evaluation level might be higher in the join tree than its
original syntactic location. That in turn affects the ph_needed
level for any contained PlaceHolderVars (that is, those PHVs had
better propagate up the join tree at least to the evaluation level
of the outer PHV). We got this mostly right, but
mark_placeholder_maybe_needed() failed to account for the effect,
and in consequence could leave the inner PHVs with ph_may_need less
than what their ultimate ph_needed value will be. That's bad
because it could lead to failure to select a join order that will
allow evaluation of the inner PHV at a valid location. Fix that,
and add an Assert that checks that we don't ever set ph_needed to
more than ph_may_need.
- Drop cheap-startup-cost paths during add_path() if we don't need
them. We can detect whether the planner top level is going to care
at all about cheap startup cost (it will only do so if
query_planner's tuple_fraction argument is greater than zero). If
it isn't, we might as well discard paths immediately whose only
advantage over others is cheap startup cost. This turns out to get
rid of quite a lot of paths in complex queries --- I saw planner
runtime reduction of more than a third on one large query. Since
add_path isn't currently passed the PlannerInfo "root", the easiest
way to tell it whether to do this was to add a bool flag to
RelOptInfo. That's a bit redundant, since all relations in a given
query level will have the same setting. But in the future it's
possible that we'd refine the control decision to work on a
per-relation basis, so this seems like a good arrangement anyway.
Per my suggestion of a few months ago.
Alvaro Herrera pushed:
- Fix thinko in comment It was confusing symlinks with hard links.
- Remove analyze_new_cluster.sh on make clean, too
- pg_upgrade: Fix exec_prog API to be less flaky. The previous
signature made it very easy to pass something other than the
printf-format specifier in the corresponding position, without any
warning from the compiler. While at it, move some of the escaping,
redirecting and quoting responsibilities from the callers into
exec_prog() itself. This makes the callsites cleaner.
- syncrep.h must include xlogdefs.h
- add #includes to plpy_subxactobject.h to make it compile standalone
- Split resowner.h. This lets files that are mere users of
ResourceOwner not automatically include the headers for stuff that
is managed by the resowner mechanism.
- Split heapam_xlog.h from heapam.h. The heapam XLog functions are
used by other modules, not all of which are interested in the rest
of the heapam API. With this, we let them get just the XLog stuff
in which they are interested and not pollute them with unrelated
includes. Also, since heapam.h no longer requires xlog.h, many
files that do include heapam.h no longer get xlog.h automatically,
including a few headers. This is useful because heapam.h is getting
pulled in by execnodes.h, which is in turn included by a lot of
- remove catcache.h from syscache.h. Instead, place a forward struct
declaration for struct catclist in syscache.h. This reduces header
- Fix assorted compilation failures in contrib. Evidently I failed to
test a compile after my earlier header shuffling.
- Split tuple struct defs from htup.h to htup_details.h. This reduces
unnecessary exposure of other headers through htup.h, which is very
widely included by many files. I have chosen to move the function
prototypes to the new file as well, because that means htup.h no
longer needs to include tupdesc.h. In itself this doesn't have much
effect in indirect inclusion of tupdesc.h throughout the tree,
because it's also required by execnodes.h; but it's something to
explore in the future, and it seemed best to do the htup.h change
now while I'm busy with it.
Robert Haas pushed:
- Back-patch fixes for some issues in our Windows socket code into
9.1. This is a backport of commit
Per discussion of bug #4958. Some of these fixes probably need to
be back-patched further, but I'm just doing this much for now.
- Document that xslt-related RPMs may be needed to build docs.
- Comment fixes. Jeff Davis, somewhat edited by me
- Fix logic bug in gistchoose and gistRelocateBuildBuffersOnSplit.
Every time the best-tuple-found-so-far changes, we need to reset all
the penalty values in which_grow to the penalties for the new best
tuple. The old code failed to do this, resulting in inferior index
quality. The original patch from Alexander Korotkov was just two
lines; I took the liberty of fleshing that out by adding a bunch of
comments that I hope will make this logic easier for others to
understand than it was for me.
- Document that COPY OUT requires an absolute pathname. As suggested
by Etsuro Fujita, but with somewhat different wording.
- Add missing period to detail message. Per note from Peter
- Document how to prevent PostgreSQL itself from exhausting memory.
The existing documentation in Linux Memory Overcommit seemed to
assume that PostgreSQL itself could never be the problem, or at
least it didn't tell you what to do about it. Per discussion with
Craig Ringer and Kevin Grittner.
- Fix checkpoint_timeout documentation to reflect current behavior.
- Cross-link to doc build requirements from install requirements.
Peter Eisentraut pushed:
- Small punctuation fixes
- Add strerror() string to chdir() error message
- Also check for Python platform-specific include directory Python can
be built to have two separate include directories: one for
platform-independent files and one for platform-specific files. So
far, this has apparently never mattered for a PL/Python build. But
with the new multi-arch Python packages in Debian and Ubuntu, this
is becoming the standard configuration on these platforms, so we
must check these directories separately to be able to build there.
Also add a bit of reporting in configure to be able to see better
what is going on with this.
- psql: Reduce compatibility warning. Only warn when connecting to a
newer server, since connecting to older servers works pretty well
nowadays. Also update the documentation a little about current
psql/server compatibility expectations.
Andrew Dunstan pushed:
- Restore setting of _USE_32BIT_TIME_T to 32 bit MSVC builds. This
was removed in commit cd004067742ee16ee63e55abfb4acbd5f09fbaab,
we're not quite sure why, but there have been reports of crashes due
to Alexander Shulgin Perl being built with it when we are not, and
it certainly seems like the right thing to do. There is still some
uncertainty as to why it sometimes fails and sometimes doesn't.
Original patch from Owais Khani, substantially reworked and extended
by Andrew Dunstan.
== Rejected Patches (for now) ==
No one was disappointed this week :-)
== Pending Patches ==
Dickson S. Guedes sent in another revision of the patch to add IF NOT
EXISTS to CREATE SCHEMA.
Amit Kapila sent in another revision of the patch to improve performance
by reducing WAL for update operations.
Alvaro Herrera sent in two more revisions of the patch to add foreign
Dimitri Fontaine sent in a patch tweaking the behavior of EVENT
Dean Rasheed sent in three more revisions of the patch to implement
Tom Lane sent in a patch to fix an issue where function parameters
aren't getting to the functions they are part of because of the lack
of type inference.
Amit Kapila sent in a patch to document pg_resetxlog and compute the
max LSN of data pages.
Owais Khan and Andrew Dunstan traded patches to use a 32-bit time_t
structure on Windows in order not to break PL/Perl.
Dimitri Fontaine sent in a patch to prevent event triggers from firing
in stand-alone mode.
Peter Eisentraut sent in a patch to change the many "%d tuples done"
lines to a sort of progress counter that stays on the same line.
Pavel Stehule sent in another revision of the patch to allow tuning
the display of booleans in psql.
Bruce Momjian and Tom Lane traded patches to make a pg_upgrade failure
mode involving socket directories less likely.
KaiGai Kohei sent in a patch to fix a case sepgsql missed in CREATE
KaiGai Kohei sent in a patch to make sepgsql follow recent changes to
the core API.
Tomas Vondra sent in two more revisions of the patch to enable pgbench
to write random samples into the transaction log.
Andrew Dunstan sent in a patch to make pg_upgrade work on Windows.
Andrew Dunstan sent in a patch to make pg_upgrade's "make check" work
on Windows with Mingw.
Tom Lane sent in a proof-of-concept patch for talking to a standalone
backend using libpq and a pair of pipes.
Sent via pgsql-announce mailing list (firstname.lastname@example.org)
To make changes to your subscription: