Home
Reading
Searching
Subscribe
Sponsors
Statistics
Posting
Contact
Spam
Lists
Links
About
Hosting
Filtering
Features Download
Marketing
Archives
FAQ
Blog
 
Gmane

From: Stefano Bagnara <apache <at> bago.org>
Subject: Re: Re: Upcoming new test-suite release
Newsgroups: gmane.mail.spam.spf.devel
Date: Wednesday 20th August 2008 17:22:57 UTC (over 10 years ago)
Julian Mehnle ha scritto:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Stefano Bagnara wrote:
>> Out last release (jSPF 0.9.6) has been done when 2007.05 was out and
>> the tests passed against 2007.05. jSPF 0.9.6 also fully pass 2007.08,
>> it has 1 issue in macromania that you added after 2007.08 in trunk.
> 
> There was no 2007.08.  Do you mean 2008.08?  If that's what you meant, 
> then there is nothing that has been added after the release of 2008.08 
> other than a cosmetic change to the spec reference field of the
> "multitxt2" (not macromania) test.

sorry, you are right.

with 2008.08 jSPF 0.9.6 fails the macromania test because %- and %_ are 
incorrectly expanded to - and _.
I fixed that as soon as I saw the macromania test in openspf testsuite.

>> Our macromania bug has been fixed in our trunk and the above report is
>> for the trunk version and your trunk testsuite including the macromania
>> test and our fix.
> 
> Usually people use only release versions, so it is important that we tell

> them which test suite release your latest release version conforms to.

jSPF 0.9.6 conforms to (and includes) rfc4408-tests-2007.05.yml
The next release will pass the current specification. The only known 
issue with jSPF 0.9.6 is the macro expansion above.

> Can the jSPF trunk version be downloaded other than by checking it out 
> from the Subversion repository?  Or is the fix included in a nightly 
> build (assuming there even is a standalone nightly build for jSPF)?

Our continuos integration server produce "lastStableBuild" artifacts:
http://hudson.zones.apache.org/hudson/view/James/job/jspf-trunk/lastStableBuild/org.apache.james.jspf$resolver/
They includes binaries and sources for the last version that passed 
every test (in this case the current trunk is working).

At ASF (Apache Software Foundation) we have a strict release process and 
we never recommend the use of snapshots and we never point users to 
nightlies or snapshots.

> If not, I suppose we should record it as follows:
> 
>   2007.05: since 0.9.6

If you tell me that it won't take 2 more years to update the page I'll 
ask a new run as soon as we'll release 0.9.7. We'll probably wait a bit 
more because we want to include newer dns libraries so to fix any 
Kaminsky DNS vulnerability and a race issue in the asynchronous version 
of our resolver.

>>>> The test above is for our latest trunk tested against the latest
>>>> yaml testsuite from your trunk.
>>> Please don't use "trunk" versions of our test suite (i.e., do not use
>>> the file named "rfc4408-tests.yml").  That's the development version.
>>>  Please use the files named "rfc4408-tests-YYYY.MM.yml" instead.
>> I use trunk in our trunk, so I can spot any issue as soon as possible.
>> In past it happened that openspf added tests that we didn't pass and we
>> complained for that and you finally removed them because they was
>> controversial, so I think it is good to test against trunk in our dev
>> snapshot.
>>
>> I'll take care to add both the latest official and the latest trunk in
>> our test tree, so that we always check both suites.
> 
> This may mislead you because sometimes we add or modify tests in the test

> suite's trunk in a temporary fashion, e.g., when an issue is still under 
> discussion.  Thus the change may get reversed later in a proper release 
> of the test suite.  It has happened before.

I know it happened. I was one of the people complaining to have it 
reverted ;-)

I check trunk by purpose because I want to be aware of this issues ASAP 
so that I can bring my opinion to this list before you release a bad 
testsuite.

> In any case, compliance statements should only ever refer to release 
> versions of the test suite

I agree.

> so trying to comply with a trunk version 
> isn't going to be of much value.

I found it invaluable in past and it is not a big cost for us.

Stefano
 
CD: 20ms