On Aug 2, 2006, at 1:49 AM, Paul Guyot wrote:

> I'm not sure I understand exactly everything you suggest.
>
> You first mention turning reads into hard errors. How does it change  
> anything? An automatic build system can still report the warnings  
> about the undeclared dependencies and call for further inspection to  
> fix the port up and then either uninstall the port considering it's  
> a failure or just proceed on activating it, allowing to test ports  
> that depend on it. Ideally, we should test ports with as many other  
> ports installed to make sure they don't test for the existence of  
> other ports.

Actually, you're combining two different suggestions into one.  The  
"hard error" suggestion was in regard to the existing checks which  
determine whether or not something is reading outside the list of  
"known dependencies", but I later suggested that perhaps the open()s  
should actually simply fail instead (return ENOENT).  The reason you  
want that to happen is because of the configure step and any runtime  
checks the port may also make during its build process for optional  
stuff.  Again, keep thinking "sandbox" in the strictest definition of  
the word.  You want to create the moral equivalent of a chroot  
environment where any stage of the ports' build/install/package  
process CANNOT see anything it's not strictly allowed to see - that's  
the only way you're going to get 3200+ ports to behave given the fact  
that ports often try to do some pretty wild stuff.

> I simply don't understand the bit about "the trigger to go off and  
> build those things first", unless you refer to the existing  
> dependency engine of DP.

I was referring to the dependency engine + the ability of your trace  
code to ALSO go and carefully record the results of each dependency's  
output, so it could be added to the manifest.   This is very similar  
to what DarwinBuild does, where it carefully records and XML (I  
believe) manifest of each and every project that's built, allowing  
true dependency tracking for subsequent projects that need projects  
built earlier.  Now, Darwin projects don't *have* explicit dependency  
information the way Ports do, forcing DarwinBuild to do the work  
instead, but the end-result is actually more controlled and  
cataloged.   What we want for Portfiles is a combination of the two  
worlds - a way of auto-detecting dependencies in order to validate the  
information in the Portfile (and aid the port author in keeping it  
correct) and a way of actually *constraining* the dependencies such  
that Ports that would otherwise glom onto purely optional stuff aren't  
allowed to do so unless explicitly allowed via a variant.

> You also mention extending the checks to files from the system  
> because of problems introduced by MacOS X updates. It's a good idea.  
> However, I think that if we can first get an automatic process that  
> tests ports on a pristine OS X install for a given version of OS X,  
> it would already solves most of the testing problem and help DP  
> developers to fix ports.

I don't think restricting access to system bits is necessarily a  
"first step", and for testing purposes it's certainly possible to to  
create "pristine OS X installs", but I don't think that's practical in  
the longer-term since people building stuff won't necessarily have the  
luxury of pristine OS X installs and, ultimately, I think the DP  
project would be better served by a mechanism which allowed package  
building to be farmed out across a distributed cluster of machines,  
many of which will be provided by volunteers and not as strictly  
controlled.  This is similar to the problem that XCode distributed  
builds have, where the remote volunteer machine can't necessarily be  
trusted 100% but the "build master" has access to all the right bits  
and can ship that to volunteer machines as necessary, ensuring that  
the end-results can be trusted.

- Jordan