Subject: Re: Re: Upcoming new test-suite release
Newsgroups: gmane.mail.spam.spf.devel
Date: 2008-08-20 17:22:57 GMT (45 weeks, 2 days, 15 hours and 19 minutes ago)
Julian Mehnle ha scritto: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Stefano Bagnara wrote: >> Out last release (jSPF 0.9.6) has been done when 2007.05 was out and >> the tests passed against 2007.05. jSPF 0.9.6 also fully pass 2007.08, >> it has 1 issue in macromania that you added after 2007.08 in trunk. > > There was no 2007.08. Do you mean 2008.08? If that's what you meant, > then there is nothing that has been added after the release of 2008.08 > other than a cosmetic change to the spec reference field of the > "multitxt2" (not macromania) test. sorry, you are right. with 2008.08 jSPF 0.9.6 fails the macromania test because %- and %_ are incorrectly expanded to - and _. I fixed that as soon as I saw the macromania test in openspf testsuite. >> Our macromania bug has been fixed in our trunk and the above report is >> for the trunk version and your trunk testsuite including the macromania >> test and our fix. > > Usually people use only release versions, so it is important that we tell > them which test suite release your latest release version conforms to. jSPF 0.9.6 conforms to (and includes) rfc4408-tests-2007.05.yml The next release will pass the current specification. The only known issue with jSPF 0.9.6 is the macro expansion above. > Can the jSPF trunk version be downloaded other than by checking it out > from the Subversion repository? Or is the fix included in a nightly > build (assuming there even is a standalone nightly build for jSPF)? Our continuos integration server produce "lastStableBuild" artifacts: http://hudson.zones.apache.org/hudson/view/James/job/jspf-trunk/lastStableBuild/org.apache.james.jspf$resolver/ They includes binaries and sources for the last version that passed every test (in this case the current trunk is working). At ASF (Apache Software Foundation) we have a strict release process and we never recommend the use of snapshots and we never point users to nightlies or snapshots. > If not, I suppose we should record it as follows: > > 2007.05: since 0.9.6 If you tell me that it won't take 2 more years to update the page I'll ask a new run as soon as we'll release 0.9.7. We'll probably wait a bit more because we want to include newer dns libraries so to fix any Kaminsky DNS vulnerability and a race issue in the asynchronous version of our resolver. >>>> The test above is for our latest trunk tested against the latest >>>> yaml testsuite from your trunk. >>> Please don't use "trunk" versions of our test suite (i.e., do not use >>> the file named "rfc4408-tests.yml"). That's the development version. >>> Please use the files named "rfc4408-tests-YYYY.MM.yml" instead. >> I use trunk in our trunk, so I can spot any issue as soon as possible. >> In past it happened that openspf added tests that we didn't pass and we >> complained for that and you finally removed them because they was >> controversial, so I think it is good to test against trunk in our dev >> snapshot. >> >> I'll take care to add both the latest official and the latest trunk in >> our test tree, so that we always check both suites. > > This may mislead you because sometimes we add or modify tests in the test > suite's trunk in a temporary fashion, e.g., when an issue is still under > discussion. Thus the change may get reversed later in a proper release > of the test suite. It has happened before. I know it happened. I was one of the people complaining to have it revertedI check trunk by purpose because I want to be aware of this issues ASAP so that I can bring my opinion to this list before you release a bad testsuite. > In any case, compliance statements should only ever refer to release > versions of the test suite I agree. > so trying to comply with a trunk version > isn't going to be of much value. I found it invaluable in past and it is not a big cost for us. Stefano