6.8.2 intermittent build failure on multiple OS X versions

[guest]

I've had multiple build failures of both 6.8.1 and 6.8.2 on OS X. This is from OS X 10.5.1 (Leopard) on Intel:

../utils/ghc-pkg/ghc-pkg-inplace update - --force-files <package.conf.inplace
WARNING: unversioned dependencies are deprecated, and will NOT be accepted by GHC 6.10: hpc bytestring template-haskell readline unix Cabal base haskell98
Reading package info from stdin ... done.
cannot find libHSghc.a on library path (ignoring)
Saving old package config file... done.
Writing new package config file... done.
Creating stage1/ghc_boot_platform.h...
Done.
touch .depend-BASE
../compiler/stage1/ghc-inplace -M -optdep-f -optdep.depend-BASE  -osuf o -I../includes   -H16m -O -I/opt/local/include -I/usr/include -L/opt/local/lib -L/usr/lib -iutils -ibasicTypes -itypes -ihsSyn -iprelude -irename -itypecheck -ideSugar -icoreSyn -ivectorise -ispecialise -isimplCore -istranal -istgSyn -isimplStg -icodeGen -imain -iprofiling -iparser -icprAnalysis -indpFlatten -iiface -icmm -inativeGen -ighci -Wall -fno-warn-name-shadowing -fno-warn-orphans -Istage2 -package hpc -package bytestring -DGHCI -package template-haskell -DGHCI_TABLES_NEXT_TO_CODE -package readline -DUSE_READLINE -cpp -fglasgow-exts -fno-generics -Rghc-timing -I. -Iparser -package unix -package Cabal -ignore-package lang -recomp -Rghc-timing -H16M '-#include "cutils.h"' -package-name ghc-6.8.2 -fgenerics basicTypes/BasicTypes.lhs basicTypes/DataCon.lhs basicTypes/Demand.lhs basicTypes/Id.lhs basicTypes/IdInfo.lhs basicTypes/Literal.lhs basicTypes/MkId.lhs basicTypes/Module.lhs basicTypes/Name.lhs basicTypes/NameEnv.lhs basicTypes/NameSet.lhs basicTypes/NewDemand.lhs basicTypes/OccName.lhs basicTypes/RdrName.lhs basicTypes/SrcLoc.lhs basicTypes/UniqSupply.lhs basicTypes/Unique.lhs basicTypes/Var.lhs basicTypes/VarEnv.lhs basicTypes/VarSet.lhs cmm/CLabel.hs cmm/Cmm.hs cmm/CmmBrokenBlock.hs cmm/CmmCPS.hs cmm/CmmCPSGen.hs cmm/CmmCallConv.hs cmm/CmmInfo.hs cmm/CmmLex.hs cmm/CmmLint.hs cmm/CmmLive.hs cmm/CmmOpt.hs cmm/CmmParse.hs cmm/CmmProcPoint.hs cmm/CmmUtils.hs cmm/Dataflow.hs cmm/MachOp.hs cmm/PprC.hs cmm/PprCmm.hs codeGen/Bitmap.hs codeGen/CgBindery.lhs codeGen/CgCallConv.hs codeGen/CgCase.lhs codeGen/CgClosure.lhs codeGen/CgCon.lhs codeGen/CgExpr.lhs codeGen/CgForeignCall.hs codeGen/CgHeapery.lhs codeGen/CgHpc.hs codeGen/CgInfoTbls.hs codeGen/CgLetNoEscape.lhs codeGen/CgMonad.lhs codeGen/CgParallel.hs codeGen/CgPrimOp.hs codeGen/CgProf.hs codeGen/CgStackery.lhs codeGen/CgTailCall.lhs codeGen/CgTicky.hs codeGen/CgUtils.hs codeGen/ClosureInfo.lhs codeGen/CodeGen.lhs codeGen/SMRep.lhs coreSyn/CoreFVs.lhs coreSyn/CoreLint.lhs coreSyn/CorePrep.lhs coreSyn/CoreSubst.lhs coreSyn/CoreSyn.lhs coreSyn/CoreTidy.lhs coreSyn/CoreUnfold.lhs coreSyn/CoreUtils.lhs coreSyn/ExternalCore.lhs coreSyn/MkExternalCore.lhs coreSyn/PprCore.lhs coreSyn/PprExternalCore.lhs cprAnalysis/CprAnalyse.lhs deSugar/Check.lhs deSugar/Coverage.lhs deSugar/Desugar.lhs deSugar/DsArrows.lhs deSugar/DsBinds.lhs deSugar/DsCCall.lhs deSugar/DsExpr.lhs deSugar/DsForeign.lhs deSugar/DsGRHSs.lhs deSugar/DsListComp.lhs deSugar/DsMeta.hs deSugar/DsMonad.lhs deSugar/DsUtils.lhs deSugar/Match.lhs deSugar/MatchCon.lhs deSugar/MatchLit.lhs ghci/ByteCodeAsm.lhs ghci/ByteCodeFFI.lhs ghci/ByteCodeGen.lhs ghci/ByteCodeInstr.lhs ghci/ByteCodeItbls.lhs ghci/ByteCodeLink.lhs ghci/Debugger.hs ghci/GhciMonad.hs ghci/GhciTags.hs ghci/InteractiveUI.hs ghci/Linker.lhs ghci/ObjLink.lhs ghci/RtClosureInspect.hs hsSyn/Convert.lhs hsSyn/HsBinds.lhs hsSyn/HsDecls.lhs hsSyn/HsDoc.hs hsSyn/HsExpr.lhs hsSyn/HsImpExp.lhs hsSyn/HsLit.lhs hsSyn/HsPat.lhs hsSyn/HsSyn.lhs hsSyn/HsTypes.lhs hsSyn/HsUtils.lhs iface/BinIface.hs iface/BuildTyCl.lhs iface/IfaceEnv.lhs iface/IfaceSyn.lhs iface/IfaceType.lhs iface/LoadIface.lhs iface/MkIface.lhs iface/TcIface.lhs main/BreakArray.hs main/CmdLineParser.hs main/CodeOutput.lhs main/Config.hs main/Constants.lhs main/DriverMkDepend.hs main/DriverPhases.hs main/DriverPipeline.hs main/DynFlags.hs main/ErrUtils.lhs main/Finder.lhs main/GHC.hs main/HeaderInfo.hs main/HscMain.lhs main/HscStats.lhs main/HscTypes.lhs main/InteractiveEval.hs main/Main.hs main/PackageConfig.hs main/Packages.lhs main/ParsePkgConf.hs main/PprTyThing.hs main/StaticFlags.hs main/SysTools.lhs main/TidyPgm.lhs nativeGen/AsmCodeGen.lhs nativeGen/GraphBase.hs nativeGen/GraphColor.hs nativeGen/GraphOps.hs nativeGen/GraphPpr.hs nativeGen/MachCodeGen.hs nativeGen/MachInstrs.hs nativeGen/MachRegs.lhs nativeGen/NCGMonad.hs nativeGen/PositionIndependentCode.hs nativeGen/PprMach.hs nativeGen/RegAllocColor.hs nativeGen/RegAllocInfo.hs nativeGen/RegAllocLinear.hs nativeGen/RegAllocStats.hs nativeGen/RegArchBase.hs nativeGen/RegArchX86.hs nativeGen/RegCoalesce.hs nativeGen/RegLiveness.hs nativeGen/RegSpill.hs nativeGen/RegSpillClean.hs nativeGen/RegSpillCost.hs ndpFlatten/FlattenInfo.hs ndpFlatten/FlattenMonad.hs ndpFlatten/Flattening.hs ndpFlatten/NDPCoreUtils.hs ndpFlatten/PArrAnal.hs parser/Ctype.lhs parser/HaddockLex.hs parser/HaddockParse.hs parser/HaddockUtils.hs parser/LexCore.hs parser/Lexer.hs parser/Parser.hs parser/ParserCore.hs parser/ParserCoreUtils.hs parser/RdrHsSyn.lhs prelude/ForeignCall.lhs prelude/PrelInfo.lhs prelude/PrelNames.lhs prelude/PrelRules.lhs prelude/PrimOp.lhs prelude/TysPrim.lhs prelude/TysWiredIn.lhs profiling/CostCentre.lhs profiling/SCCfinal.lhs rename/RnBinds.lhs rename/RnEnv.lhs rename/RnExpr.lhs rename/RnHsDoc.hs rename/RnHsSyn.lhs rename/RnNames.lhs rename/RnSource.lhs rename/RnTypes.lhs simplCore/CSE.lhs simplCore/FloatIn.lhs simplCore/FloatOut.lhs simplCore/LiberateCase.lhs simplCore/OccurAnal.lhs simplCore/SAT.lhs simplCore/SATMonad.lhs simplCore/SetLevels.lhs simplCore/SimplCore.lhs simplCore/SimplEnv.lhs simplCore/SimplMonad.lhs simplCore/SimplUtils.lhs simplCore/Simplify.lhs simplStg/SRT.lhs simplStg/SimplStg.lhs simplStg/StgStats.lhs specialise/Rules.lhs specialise/SpecConstr.lhs specialise/Specialise.lhs stgSyn/CoreToStg.lhs stgSyn/StgLint.lhs stgSyn/StgSyn.lhs stranal/DmdAnal.lhs stranal/SaAbsInt.lhs stranal/SaLib.lhs stranal/StrictAnal.lhs stranal/WorkWrap.lhs stranal/WwLib.lhs typecheck/FamInst.lhs typecheck/Inst.lhs typecheck/TcArrows.lhs typecheck/TcBinds.lhs typecheck/TcClassDcl.lhs typecheck/TcDefaults.lhs typecheck/TcDeriv.lhs typecheck/TcEnv.lhs typecheck/TcExpr.lhs typecheck/TcForeign.lhs typecheck/TcGadt.lhs typecheck/TcGenDeriv.lhs typecheck/TcHsSyn.lhs typecheck/TcHsType.lhs typecheck/TcInstDcls.lhs typecheck/TcMType.lhs typecheck/TcMatches.lhs typecheck/TcPat.lhs typecheck/TcRnDriver.lhs typecheck/TcRnMonad.lhs typecheck/TcRnTypes.lhs typecheck/TcRules.lhs typecheck/TcSimplify.lhs typecheck/TcSplice.lhs typecheck/TcTyClsDecls.lhs typecheck/TcTyDecls.lhs typecheck/TcTyFuns.lhs typecheck/TcType.lhs typecheck/TcUnify.lhs types/Class.lhs types/Coercion.lhs types/FamInstEnv.lhs types/FunDeps.lhs types/Generics.lhs types/InstEnv.lhs types/TyCon.lhs types/Type.lhs types/TypeRep.lhs types/Unify.lhs utils/Bag.lhs utils/Binary.hs utils/BufWrite.hs utils/Digraph.lhs utils/Encoding.hs utils/FastMutInt.lhs utils/FastString.lhs utils/FastTypes.lhs utils/FiniteMap.lhs utils/IOEnv.hs utils/ListSetOps.lhs utils/Maybes.lhs utils/OrdList.lhs utils/Outputable.lhs utils/Panic.lhs utils/Pretty.lhs utils/State.hs utils/StringBuffer.lhs utils/UniqFM.lhs utils/UniqSet.lhs utils/Util.lhs vectorise/VectBuiltIn.hs vectorise/VectCore.hs vectorise/VectMonad.hs vectorise/VectType.hs vectorise/VectUtils.hs vectorise/Vectorise.hs
.depend-BASE: openFile: resource busy (file is locked)
<<ghc: 472175192 bytes, 74 GCs, 4900454/12947456 avg/max bytes residency (5 samples), 27M in use, 0.00 INIT (0.00 elapsed), 0.82 MUT (7.18 elapsed), 0.13 GC (0.15 elapsed) :ghc>>
make[2]: *** [depend] Error 1
make[1]: *** [stage2] Error 2
make: *** [bootstrap2] Error 2

The above was building 6.8.2 using a 6.8.2 bootstrap compiler.

I have also seen a similar error building 6.8.1 using 6.4 on OS X 10.4.11 (Tiger) PPC. In that case, the error came from openTempFile in Cabal's Utils.hs. The code seemed correct, but the error was the same as you'd get trying to delete a file before closing it.

The error does not happen every time. My guess is that there is some kind of race condition in accessing, closing or deleting files.

[gw]

My earlier guess about this being a race condition might be wrong. I see now that file locking was rewritten for 6.8. Could just be an error in the code. I'll print the details from lockFile and unlockFile and see if that leads anywhere.

[gw]

Since this problem has shown up intermittently, I swapped memory sticks in my laptop with a spare set. The bug is still there. So it's not the hardware.

[igloo]

Thanks for the report. Does running

../compiler/stage1/ghc-inplace -M -optdep-f -optdep.depend-BASE  -osuf o -I../includes   -H16m -O -I/opt/local/include -I/usr/include -L/opt/local/lib -L/usr/lib -iutils -ibasicTypes -itypes -ihsSyn -iprelude -irename -itypecheck -ideSugar -icoreSyn -ivectorise -ispecialise -isimplCore -istranal -istgSyn -isimplStg -icodeGen -imain -iprofiling -iparser -icprAnalysis -indpFlatten -iiface -icmm -inativeGen -ighci -Wall -fno-warn-name-shadowing -fno-warn-orphans -Istage2 -package hpc -package bytestring -DGHCI -package template-haskell -DGHCI_TABLES_NEXT_TO_CODE -package readline -DUSE_READLINE -cpp -fglasgow-exts -fno-generics -Rghc-timing -I. -Iparser -package unix -package Cabal -ignore-package lang -recomp -Rghc-timing -H16M '-#include "cutils.h"' -package-name ghc-6.8.2 -fgenerics basicTypes/BasicTypes.lhs basicTypes/DataCon.lhs basicTypes/Demand.lhs basicTypes/Id.lhs basicTypes/IdInfo.lhs basicTypes/Literal.lhs basicTypes/MkId.lhs basicTypes/Module.lhs basicTypes/Name.lhs basicTypes/NameEnv.lhs basicTypes/NameSet.lhs basicTypes/NewDemand.lhs basicTypes/OccName.lhs basicTypes/RdrName.lhs basicTypes/SrcLoc.lhs basicTypes/UniqSupply.lhs basicTypes/Unique.lhs basicTypes/Var.lhs basicTypes/VarEnv.lhs basicTypes/VarSet.lhs cmm/CLabel.hs cmm/Cmm.hs cmm/CmmBrokenBlock.hs cmm/CmmCPS.hs cmm/CmmCPSGen.hs cmm/CmmCallConv.hs cmm/CmmInfo.hs cmm/CmmLex.hs cmm/CmmLint.hs cmm/CmmLive.hs cmm/CmmOpt.hs cmm/CmmParse.hs cmm/CmmProcPoint.hs cmm/CmmUtils.hs cmm/Dataflow.hs cmm/MachOp.hs cmm/PprC.hs cmm/PprCmm.hs codeGen/Bitmap.hs codeGen/CgBindery.lhs codeGen/CgCallConv.hs codeGen/CgCase.lhs codeGen/CgClosure.lhs codeGen/CgCon.lhs codeGen/CgExpr.lhs codeGen/CgForeignCall.hs codeGen/CgHeapery.lhs codeGen/CgHpc.hs codeGen/CgInfoTbls.hs codeGen/CgLetNoEscape.lhs codeGen/CgMonad.lhs codeGen/CgParallel.hs codeGen/CgPrimOp.hs codeGen/CgProf.hs codeGen/CgStackery.lhs codeGen/CgTailCall.lhs codeGen/CgTicky.hs codeGen/CgUtils.hs codeGen/ClosureInfo.lhs codeGen/CodeGen.lhs codeGen/SMRep.lhs coreSyn/CoreFVs.lhs coreSyn/CoreLint.lhs coreSyn/CorePrep.lhs coreSyn/CoreSubst.lhs coreSyn/CoreSyn.lhs coreSyn/CoreTidy.lhs coreSyn/CoreUnfold.lhs coreSyn/CoreUtils.lhs coreSyn/ExternalCore.lhs coreSyn/MkExternalCore.lhs coreSyn/PprCore.lhs coreSyn/PprExternalCore.lhs cprAnalysis/CprAnalyse.lhs deSugar/Check.lhs deSugar/Coverage.lhs deSugar/Desugar.lhs deSugar/DsArrows.lhs deSugar/DsBinds.lhs deSugar/DsCCall.lhs deSugar/DsExpr.lhs deSugar/DsForeign.lhs deSugar/DsGRHSs.lhs deSugar/DsListComp.lhs deSugar/DsMeta.hs deSugar/DsMonad.lhs deSugar/DsUtils.lhs deSugar/Match.lhs deSugar/MatchCon.lhs deSugar/MatchLit.lhs ghci/ByteCodeAsm.lhs ghci/ByteCodeFFI.lhs ghci/ByteCodeGen.lhs ghci/ByteCodeInstr.lhs ghci/ByteCodeItbls.lhs ghci/ByteCodeLink.lhs ghci/Debugger.hs ghci/GhciMonad.hs ghci/GhciTags.hs ghci/InteractiveUI.hs ghci/Linker.lhs ghci/ObjLink.lhs ghci/RtClosureInspect.hs hsSyn/Convert.lhs hsSyn/HsBinds.lhs hsSyn/HsDecls.lhs hsSyn/HsDoc.hs hsSyn/HsExpr.lhs hsSyn/HsImpExp.lhs hsSyn/HsLit.lhs hsSyn/HsPat.lhs hsSyn/HsSyn.lhs hsSyn/HsTypes.lhs hsSyn/HsUtils.lhs iface/BinIface.hs iface/BuildTyCl.lhs iface/IfaceEnv.lhs iface/IfaceSyn.lhs iface/IfaceType.lhs iface/LoadIface.lhs iface/MkIface.lhs iface/TcIface.lhs main/BreakArray.hs main/CmdLineParser.hs main/CodeOutput.lhs main/Config.hs main/Constants.lhs main/DriverMkDepend.hs main/DriverPhases.hs main/DriverPipeline.hs main/DynFlags.hs main/ErrUtils.lhs main/Finder.lhs main/GHC.hs main/HeaderInfo.hs main/HscMain.lhs main/HscStats.lhs main/HscTypes.lhs main/InteractiveEval.hs main/Main.hs main/PackageConfig.hs main/Packages.lhs main/ParsePkgConf.hs main/PprTyThing.hs main/StaticFlags.hs main/SysTools.lhs main/TidyPgm.lhs nativeGen/AsmCodeGen.lhs nativeGen/GraphBase.hs nativeGen/GraphColor.hs nativeGen/GraphOps.hs nativeGen/GraphPpr.hs nativeGen/MachCodeGen.hs nativeGen/MachInstrs.hs nativeGen/MachRegs.lhs nativeGen/NCGMonad.hs nativeGen/PositionIndependentCode.hs nativeGen/PprMach.hs nativeGen/RegAllocColor.hs nativeGen/RegAllocInfo.hs nativeGen/RegAllocLinear.hs nativeGen/RegAllocStats.hs nativeGen/RegArchBase.hs nativeGen/RegArchX86.hs nativeGen/RegCoalesce.hs nativeGen/RegLiveness.hs nativeGen/RegSpill.hs nativeGen/RegSpillClean.hs nativeGen/RegSpillCost.hs ndpFlatten/FlattenInfo.hs ndpFlatten/FlattenMonad.hs ndpFlatten/Flattening.hs ndpFlatten/NDPCoreUtils.hs ndpFlatten/PArrAnal.hs parser/Ctype.lhs parser/HaddockLex.hs parser/HaddockParse.hs parser/HaddockUtils.hs parser/LexCore.hs parser/Lexer.hs parser/Parser.hs parser/ParserCore.hs parser/ParserCoreUtils.hs parser/RdrHsSyn.lhs prelude/ForeignCall.lhs prelude/PrelInfo.lhs prelude/PrelNames.lhs prelude/PrelRules.lhs prelude/PrimOp.lhs prelude/TysPrim.lhs prelude/TysWiredIn.lhs profiling/CostCentre.lhs profiling/SCCfinal.lhs rename/RnBinds.lhs rename/RnEnv.lhs rename/RnExpr.lhs rename/RnHsDoc.hs rename/RnHsSyn.lhs rename/RnNames.lhs rename/RnSource.lhs rename/RnTypes.lhs simplCore/CSE.lhs simplCore/FloatIn.lhs simplCore/FloatOut.lhs simplCore/LiberateCase.lhs simplCore/OccurAnal.lhs simplCore/SAT.lhs simplCore/SATMonad.lhs simplCore/SetLevels.lhs simplCore/SimplCore.lhs simplCore/SimplEnv.lhs simplCore/SimplMonad.lhs simplCore/SimplUtils.lhs simplCore/Simplify.lhs simplStg/SRT.lhs simplStg/SimplStg.lhs simplStg/StgStats.lhs specialise/Rules.lhs specialise/SpecConstr.lhs specialise/Specialise.lhs stgSyn/CoreToStg.lhs stgSyn/StgLint.lhs stgSyn/StgSyn.lhs stranal/DmdAnal.lhs stranal/SaAbsInt.lhs stranal/SaLib.lhs stranal/StrictAnal.lhs stranal/WorkWrap.lhs stranal/WwLib.lhs typecheck/FamInst.lhs typecheck/Inst.lhs typecheck/TcArrows.lhs typecheck/TcBinds.lhs typecheck/TcClassDcl.lhs typecheck/TcDefaults.lhs typecheck/TcDeriv.lhs typecheck/TcEnv.lhs typecheck/TcExpr.lhs typecheck/TcForeign.lhs typecheck/TcGadt.lhs typecheck/TcGenDeriv.lhs typecheck/TcHsSyn.lhs typecheck/TcHsType.lhs typecheck/TcInstDcls.lhs typecheck/TcMType.lhs typecheck/TcMatches.lhs typecheck/TcPat.lhs typecheck/TcRnDriver.lhs typecheck/TcRnMonad.lhs typecheck/TcRnTypes.lhs typecheck/TcRules.lhs typecheck/TcSimplify.lhs typecheck/TcSplice.lhs typecheck/TcTyClsDecls.lhs typecheck/TcTyDecls.lhs typecheck/TcTyFuns.lhs typecheck/TcType.lhs typecheck/TcUnify.lhs types/Class.lhs types/Coercion.lhs types/FamInstEnv.lhs types/FunDeps.lhs types/Generics.lhs types/InstEnv.lhs types/TyCon.lhs types/Type.lhs types/TypeRep.lhs types/Unify.lhs utils/Bag.lhs utils/Binary.hs utils/BufWrite.hs utils/Digraph.lhs utils/Encoding.hs utils/FastMutInt.lhs utils/FastString.lhs utils/FastTypes.lhs utils/FiniteMap.lhs utils/IOEnv.hs utils/ListSetOps.lhs utils/Maybes.lhs utils/OrdList.lhs utils/Outputable.lhs utils/Panic.lhs utils/Pretty.lhs utils/State.hs utils/StringBuffer.lhs utils/UniqFM.lhs utils/UniqSet.lhs utils/Util.lhs vectorise/VectBuiltIn.hs vectorise/VectCore.hs vectorise/VectMonad.hs vectorise/VectType.hs vectorise/VectUtils.hs vectorise/Vectorise.hs

by hand in compiler/ reproduce the problem? It sounds like printing info from the internals of openFile, as you suggest, or looking at the equivalent of strace's output (ktrace on OS X, if I remember correctly), should shed some light.

It's curious that I haven't seen anyone else report this. What sort of filesystem are you working on?

[gwright]

I suspect that this bug has been seen before. In early December, Duncan Coutts sent a message to the Cabal list saying that he had fixed a problem in which the ghc's version wasn't detected. I saw the same thing when building 6.8.1 on Panther/ppc. It was caused by a lock error on the temporary file that received the output of ghc --numeric-version. (Cabal had caught the error and discarded it, returning "" for the version string.) Duncan's patch was to move the temporary file from the current directory to that returned by getTemporaryDirectory. I tried a similar thing --- I just hard coded "/tmp" as the temp file's directory --- but still got lock errors.

But it's been difficult to make this bug consistently reproducible. The most repeatable setup has been to build in an eshell or just a comint buffer running under Aquamacs. This fails almost every time with a linker error complaining that libHShaskell98-1.0.1.0.a doesn't have a table of contents. It looks like ranlib wasn't run on it. Strangely, vanilla emacs 22.1 is OK. Building under MacPorts? fails about a quarter of the time, usually with a lock error. MacPorts? runs it scripts under a tclsh, but I've not been able to reproduce the bug running under a bare tclsh.

I had thought that this might be related to the fix for bug #724, which avoids putting file descriptors inherited from external processes into nonblocking mode. I backed out the change and built a new bootstrap compiler (using MacPorts?), then built in an eshell under Aquamacs a fresh tree that had the #724 change backed out. No dice. The build still failed with a link error because ranlib had not been run on libHShaskell98-1.0.1.0.a.

Here's one thing that may or may not be significant: when libHShaskell-98 is registered, the build logs under Aquamacs and bash/terminal show a different order of operations. Under bash/terminal,

ar: creating archive dist/build/libHShaskell98-1.0.1.0.a
/usr/bin/ld -x -r -o dist/build/HShaskell98-1.0.1.0.o -L/opt/local/lib dist/build/Array.o dist/build/CPUTime.o dist/build/Char.o dist/build/Complex.o dist/build/Directory.o dist/build/IO.o dist/build/Ix.o dist/build/List.o dist/build/Locale.o dist/build/Maybe.o dist/build/Monad.o dist/build/Random.o dist/build/Ratio.o dist/build/System.o dist/build/Time.o dist/build/Bits.o dist/build/CError.o dist/build/CForeign.o dist/build/CString.o dist/build/CTypes.o dist/build/ForeignPtr.o dist/build/Int.o dist/build/MarshalAlloc.o dist/build/MarshalArray.o dist/build/MarshalError.o dist/build/MarshalUtils.o dist/build/Ptr.o dist/build/StablePtr.o dist/build/Storable.o dist/build/Word.o `find dist/build -name "*_stub.o" -print` 
Reading package info from "dist/inplace-pkg-config" ... done.
Saving old package config file... done.
Writing new package config file... done.
Registering haskell98-1.0.1.0...
create dist/inplace-pkg-config
("../../utils/ghc-pkg/ghc-pkg-inplace",["update","dist/inplace-pkg-config"])

while Aquamacs shows

ar: creating archive dist/build/libHShaskell98-1.0.1.0.a
/usr/bin/ld -x -r -o dist/build/HShaskell98-1.0.1.0.o -L/opt/local/lib dist/build/Array.o dist/build/CPUTime.o dist/build/Char.o dist/build/Complex.o dist/build/Directory.o dist/build/IO.o dist/build/Ix.o dist/build/List.o dist/build/Locale.o dist/build/Maybe.o dist/build/Monad.o dist/build/Random.o dist/build/Ratio.o dist/build/System.o dist/build/Time.o dist/build/Bits.o dist/build/CError.o dist/build/CForeign.o dist/build/CString.o dist/build/CTypes.o dist/build/ForeignPtr.o dist/build/Int.o dist/build/MarshalAlloc.o dist/build/MarshalArray.o dist/build/MarshalError.o dist/build/MarshalUtils.o dist/build/Ptr.o dist/build/StablePtr.o dist/build/Storable.o dist/build/Word.o `find dist/build -name "*_stub.o" -print` 
Registering haskell98-1.0.1.0...
create dist/inplace-pkg-config
("../../utils/ghc-pkg/ghc-pkg-inplace",["update","dist/inplace-pkg-config"])
Reading package info from "dist/inplace-pkg-config" ... done.
Saving old package config file... done.
Writing new package config file... done.

and the build fails shortly afterward when stage2 begins because libHSHaskell98.a has no table of contents. Ranlib should have been run by installLib (in Cabal/Distribution/Simple/GHC.hs), but for some reason it wasn't. But the different order of output may not mean anything. When I build under vanilla emacs I get the same order of output as under Aquamacs, but the build succeeds.

I modified installLib to force the verbosity of calls to rawSystemProgram to "deafening". This should have at least printed out when ranlib was called. In this case, the build failed with a lock error:

Creating main/Config.hs ... 
done.
Creating stage1/ghc_boot_platform.h...
Done.
gcc -E -I/opt/local/include  -undef -traditional -P -I../includes    -x c prelude/primops.txt.pp | \
	grep -v '^#pragma GCC' > prelude/primops.txt
../utils/genprimopcode/genprimopcode --data-decl          < prelude/primops.txt > primop-data-decl.hs-incl
../utils/genprimopcode/genprimopcode --primop-tag         < prelude/primops.txt > primop-tag.hs-incl
../utils/genprimopcode/genprimopcode --primop-list        < prelude/primops.txt > primop-list.hs-incl
../utils/genprimopcode/genprimopcode --has-side-effects   < prelude/primops.txt > primop-has-side-effects.hs-incl
../utils/genprimopcode/genprimopcode --out-of-line        < prelude/primops.txt > primop-out-of-line.hs-incl
../utils/genprimopcode/genprimopcode --commutable         < prelude/primops.txt > primop-commutable.hs-incl
../utils/genprimopcode/genprimopcode --needs-wrapper      < prelude/primops.txt > primop-needs-wrapper.hs-incl
../utils/genprimopcode/genprimopcode --can-fail           < prelude/primops.txt > primop-can-fail.hs-incl
../utils/genprimopcode/genprimopcode --strictness         < prelude/primops.txt > primop-strictness.hs-incl
../utils/genprimopcode/genprimopcode --primop-primop-info < prelude/primops.txt > primop-primop-info.hs-incl
touch .depend-BASE
/opt/local/bin/ghc -M -optdep-f -optdep.depend-BASE  -osuf o -I../includes -optdep--exclude-module=Compat.RawSystem -optdep--exclude-module=Compat.Directory -optdep--exclude-module=Compat.Unicode -optdep--exclude-module=Distribution.Compat.FilePath -optdep--exclude-module=Distribution.Compat.ReadP -optdep--exclude-module=Distribution.Extension -optdep--exclude-module=Distribution.GetOpt -optdep--exclude-module=Distribution.InstalledPackageInfo -optdep--exclude-module=Distribution.License -optdep--exclude-module=Distribution.Package -optdep--exclude-module=Distribution.ParseUtils -optdep--exclude-module=Distribution.Compiler -optdep--exclude-module=Distribution.Version -optdep--exclude-module=System.FilePath -optdep--exclude-module=System.FilePath.Posix -optdep--exclude-module=System.FilePath.Windows -optdep--exclude-module=System.Directory.Internals -optdep--exclude-module=Trace.Hpc.Mix -optdep--exclude-module=Trace.Hpc.Tix -optdep--exclude-module=Trace.Hpc.Util   -H128m -O -fasm -Rghc-timing -I/opt/local/include -L/opt/local/lib -iutils -ibasicTypes -itypes -ihsSyn -iprelude -irename -itypecheck -ideSugar -icoreSyn -ivectorise -ispecialise -isimplCore -istranal -istgSyn -isimplStg -icodeGen -imain -iprofiling -iparser -icprAnalysis -indpFlatten -iiface -icmm -inativeGen -Wall -fno-warn-name-shadowing -fno-warn-orphans -Istage1 -cpp -fglasgow-exts -fno-generics -Rghc-timing -I. -Iparser -package unix -ignore-package lang -recomp -Rghc-timing -O0 -DDEBUG -W -H16M '-#include "cutils.h"' -DUSING_COMPAT -i../compat -ignore-package Cabal -package directory -package pretty -package containers basicTypes/BasicTypes.lhs basicTypes/DataCon.lhs basicTypes/Demand.lhs basicTypes/Id.lhs basicTypes/IdInfo.lhs basicTypes/Literal.lhs basicTypes/MkId.lhs basicTypes/Module.lhs basicTypes/Name.lhs basicTypes/NameEnv.lhs basicTypes/NameSet.lhs basicTypes/NewDemand.lhs basicTypes/OccName.lhs basicTypes/RdrName.lhs basicTypes/SrcLoc.lhs basicTypes/UniqSupply.lhs basicTypes/Unique.lhs basicTypes/Var.lhs basicTypes/VarEnv.lhs basicTypes/VarSet.lhs cmm/CLabel.hs cmm/Cmm.hs cmm/CmmBrokenBlock.hs cmm/CmmCPS.hs cmm/CmmCPSGen.hs cmm/CmmCallConv.hs cmm/CmmInfo.hs cmm/CmmLex.hs cmm/CmmLint.hs cmm/CmmLive.hs cmm/CmmOpt.hs cmm/CmmParse.hs cmm/CmmProcPoint.hs cmm/CmmUtils.hs cmm/Dataflow.hs cmm/MachOp.hs cmm/PprC.hs cmm/PprCmm.hs codeGen/Bitmap.hs codeGen/CgBindery.lhs codeGen/CgCallConv.hs codeGen/CgCase.lhs codeGen/CgClosure.lhs codeGen/CgCon.lhs codeGen/CgExpr.lhs codeGen/CgForeignCall.hs codeGen/CgHeapery.lhs codeGen/CgHpc.hs codeGen/CgInfoTbls.hs codeGen/CgLetNoEscape.lhs codeGen/CgMonad.lhs codeGen/CgParallel.hs codeGen/CgPrimOp.hs codeGen/CgProf.hs codeGen/CgStackery.lhs codeGen/CgTailCall.lhs codeGen/CgTicky.hs codeGen/CgUtils.hs codeGen/ClosureInfo.lhs codeGen/CodeGen.lhs codeGen/SMRep.lhs coreSyn/CoreFVs.lhs coreSyn/CoreLint.lhs coreSyn/CorePrep.lhs coreSyn/CoreSubst.lhs coreSyn/CoreSyn.lhs coreSyn/CoreTidy.lhs coreSyn/CoreUnfold.lhs coreSyn/CoreUtils.lhs coreSyn/ExternalCore.lhs coreSyn/MkExternalCore.lhs coreSyn/PprCore.lhs coreSyn/PprExternalCore.lhs cprAnalysis/CprAnalyse.lhs deSugar/Check.lhs deSugar/Coverage.lhs deSugar/Desugar.lhs deSugar/DsArrows.lhs deSugar/DsBinds.lhs deSugar/DsCCall.lhs deSugar/DsExpr.lhs deSugar/DsForeign.lhs deSugar/DsGRHSs.lhs deSugar/DsListComp.lhs deSugar/DsMonad.lhs deSugar/DsUtils.lhs deSugar/Match.lhs deSugar/MatchCon.lhs deSugar/MatchLit.lhs hsSyn/HsBinds.lhs hsSyn/HsDecls.lhs hsSyn/HsDoc.hs hsSyn/HsExpr.lhs hsSyn/HsImpExp.lhs hsSyn/HsLit.lhs hsSyn/HsPat.lhs hsSyn/HsSyn.lhs hsSyn/HsTypes.lhs hsSyn/HsUtils.lhs iface/BinIface.hs iface/BuildTyCl.lhs iface/IfaceEnv.lhs iface/IfaceSyn.lhs iface/IfaceType.lhs iface/LoadIface.lhs iface/MkIface.lhs iface/TcIface.lhs main/BreakArray.hs main/CmdLineParser.hs main/CodeOutput.lhs main/Config.hs main/Constants.lhs main/DriverMkDepend.hs main/DriverPhases.hs main/DriverPipeline.hs main/DynFlags.hs main/ErrUtils.lhs main/Finder.lhs main/GHC.hs main/HeaderInfo.hs main/HscMain.lhs main/HscStats.lhs main/HscTypes.lhs main/InteractiveEval.hs main/Main.hs main/PackageConfig.hs main/Packages.lhs main/ParsePkgConf.hs main/PprTyThing.hs main/StaticFlags.hs main/SysTools.lhs main/TidyPgm.lhs nativeGen/AsmCodeGen.lhs nativeGen/GraphBase.hs nativeGen/GraphColor.hs nativeGen/GraphOps.hs nativeGen/GraphPpr.hs nativeGen/MachCodeGen.hs nativeGen/MachInstrs.hs nativeGen/MachRegs.lhs nativeGen/NCGMonad.hs nativeGen/PositionIndependentCode.hs nativeGen/PprMach.hs nativeGen/RegAllocColor.hs nativeGen/RegAllocInfo.hs nativeGen/RegAllocLinear.hs nativeGen/RegAllocStats.hs nativeGen/RegArchBase.hs nativeGen/RegArchX86.hs nativeGen/RegCoalesce.hs nativeGen/RegLiveness.hs nativeGen/RegSpill.hs nativeGen/RegSpillClean.hs nativeGen/RegSpillCost.hs ndpFlatten/FlattenInfo.hs ndpFlatten/FlattenMonad.hs ndpFlatten/Flattening.hs ndpFlatten/NDPCoreUtils.hs ndpFlatten/PArrAnal.hs parser/Ctype.lhs parser/HaddockLex.hs parser/HaddockParse.hs parser/HaddockUtils.hs parser/LexCore.hs parser/Lexer.hs parser/Parser.hs parser/ParserCore.hs parser/ParserCoreUtils.hs parser/RdrHsSyn.lhs prelude/ForeignCall.lhs prelude/PrelInfo.lhs prelude/PrelNames.lhs prelude/PrelRules.lhs prelude/PrimOp.lhs prelude/TysPrim.lhs prelude/TysWiredIn.lhs profiling/CostCentre.lhs profiling/SCCfinal.lhs rename/RnBinds.lhs rename/RnEnv.lhs rename/RnExpr.lhs rename/RnHsDoc.hs rename/RnHsSyn.lhs rename/RnNames.lhs rename/RnSource.lhs rename/RnTypes.lhs simplCore/CSE.lhs simplCore/FloatIn.lhs simplCore/FloatOut.lhs simplCore/LiberateCase.lhs simplCore/OccurAnal.lhs simplCore/SAT.lhs simplCore/SATMonad.lhs simplCore/SetLevels.lhs simplCore/SimplCore.lhs simplCore/SimplEnv.lhs simplCore/SimplMonad.lhs simplCore/SimplUtils.lhs simplCore/Simplify.lhs simplStg/SRT.lhs simplStg/SimplStg.lhs simplStg/StgStats.lhs specialise/Rules.lhs specialise/SpecConstr.lhs specialise/Specialise.lhs stgSyn/CoreToStg.lhs stgSyn/StgLint.lhs stgSyn/StgSyn.lhs stranal/DmdAnal.lhs stranal/SaAbsInt.lhs stranal/SaLib.lhs stranal/StrictAnal.lhs stranal/WorkWrap.lhs stranal/WwLib.lhs typecheck/FamInst.lhs typecheck/Inst.lhs typecheck/TcArrows.lhs typecheck/TcBinds.lhs typecheck/TcClassDcl.lhs typecheck/TcDefaults.lhs typecheck/TcDeriv.lhs typecheck/TcEnv.lhs typecheck/TcExpr.lhs typecheck/TcForeign.lhs typecheck/TcGadt.lhs typecheck/TcGenDeriv.lhs typecheck/TcHsSyn.lhs typecheck/TcHsType.lhs typecheck/TcInstDcls.lhs typecheck/TcMType.lhs typecheck/TcMatches.lhs typecheck/TcPat.lhs typecheck/TcRnDriver.lhs typecheck/TcRnMonad.lhs typecheck/TcRnTypes.lhs typecheck/TcRules.lhs typecheck/TcSimplify.lhs typecheck/TcTyClsDecls.lhs typecheck/TcTyDecls.lhs typecheck/TcTyFuns.lhs typecheck/TcType.lhs typecheck/TcUnify.lhs types/Class.lhs types/Coercion.lhs types/FamInstEnv.lhs types/FunDeps.lhs types/Generics.lhs types/InstEnv.lhs types/TyCon.lhs types/Type.lhs types/TypeRep.lhs types/Unify.lhs utils/Bag.lhs utils/Binary.hs utils/BufWrite.hs utils/Digraph.lhs utils/Encoding.hs utils/FastMutInt.lhs utils/FastString.lhs utils/FastTypes.lhs utils/FiniteMap.lhs utils/IOEnv.hs utils/ListSetOps.lhs utils/Maybes.lhs utils/OrdList.lhs utils/Outputable.lhs utils/Panic.lhs utils/Pretty.lhs utils/State.hs utils/StringBuffer.lhs utils/UniqFM.lhs utils/UniqSet.lhs utils/Util.lhs vectorise/VectBuiltIn.hs vectorise/VectCore.hs vectorise/VectMonad.hs vectorise/VectType.hs vectorise/VectUtils.hs vectorise/Vectorise.hs
main/InteractiveEval.hs: openBinaryFile: resource busy (file is locked)
<<ghc: 167304936 bytes, 21 GCs, 1953792/3670016 avg/max bytes residency (3 samples), 19M in use, 0.01 INIT (0.00 elapsed), 0.23 MUT (2.65 elapsed), 0.04 GC (0.05 elapsed) :ghc>>
make[1]: *** [depend] Error 1
make: *** [stage1] Error 1

Just like playing whack-a-mole. The bug has now moved somewhere else.

The common factor may be process handing. I've been trying to use dtrace to trace process (alas, ktrace is gone on Leopard). Unfortunately, on OS X, dtrace is at best beta quality. For example, you can not reliably get at least some of the arguments to a process that is execve'd as you can on Solaris.

Since the error I can easily reproduce is the missed ranlib, I will keep working with dtrace to try to get some kind of usable trace. I am travelling and only have my MacBook? running Leopard, but next week I will have access to my wife's iBook running Panther. I can do a ktrace with hers.

The debugging process has been slow since I'm building 6.8.2 with a 6.8.2 bootstrap compiler. When I make a patch, I rebuild the bootstrap compiler until it succeeds, then do a test build. By keeping the bootstrap compiler tree identical to the test tree I hope to avoid chasing my tail with a buggy bootstrap. But if there are any suggestions to safely speed up the process, I would welcome them.

Any hints as to what might have changed from 6.6 to 6.8 that might cause this problem?

[gw]

Some progress:

There are two distinct bugs here. The problem with apparently failed ranlibs (or ld not determining if an archive has a table of contents) has been fixed by Apple's 10.5.2 update. I guess this is what the release note item "improved stability for third party applications" meant. This bug was only seen on Leopard.

This leaves only the locking bug. A Macports build of 6.8.2 still fails because .depend-BASE appears to be locked (perhaps a race that allows the file descriptor to be reused by ghc before the file is unlocked):

../utils/ghc-pkg/ghc-pkg-inplace update - --force-files <package.conf.inplace
WARNING: unversioned dependencies are deprecated, and will NOT be accepted by GHC 6.10: hpc bytestring template-haskell readline unix Cabal base haskell98
Reading package info from stdin ... done.
cannot find libHSghc.a on library path (ignoring)
Saving old package config file... done.
Writing new package config file... done.
Creating stage1/ghc_boot_platform.h...
Done.
touch .depend-BASE
../compiler/stage1/ghc-inplace -M -optdep-f -optdep.depend-BASE  -osuf o -I../includes   -H16m -O -I/opt/local/include -L/opt/local/lib -iutils -ibasicTypes -itypes -ihsSyn -iprelude -irename -itypecheck -ideSugar -icoreSyn -ivectorise -ispecialise -isimplCore -istranal -istgSyn -isimplStg -icodeGen -imain -iprofiling -iparser -icprAnalysis -indpFlatten -iiface -icmm -inativeGen -ighci -Wall -fno-warn-name-shadowing -fno-warn-orphans -Istage2 -package hpc -package bytestring -DGHCI -package template-haskell -DGHCI_TABLES_NEXT_TO_CODE -package readline -DUSE_READLINE -cpp -fglasgow-exts -fno-generics -Rghc-timing -I. -Iparser -package unix -package Cabal -ignore-package lang -recomp -Rghc-timing -H16M '-#include "cutils.h"' -package-name ghc-6.8.2 -fgenerics basicTypes/BasicTypes.lhs basicTypes/DataCon.lhs basicTypes/Demand.lhs basicTypes/Id.lhs basicTypes/IdInfo.lhs basicTypes/Literal.lhs basicTypes/MkId.lhs basicTypes/Module.lhs basicTypes/Name.lhs basicTypes/NameEnv.lhs basicTypes/NameSet.lhs basicTypes/NewDemand.lhs basicTypes/OccName.lhs basicTypes/RdrName.lhs basicTypes/SrcLoc.lhs basicTypes/UniqSupply.lhs basicTypes/Unique.lhs basicTypes/Var.lhs basicTypes/VarEnv.lhs basicTypes/VarSet.lhs cmm/CLabel.hs cmm/Cmm.hs cmm/CmmBrokenBlock.hs cmm/CmmCPS.hs cmm/CmmCPSGen.hs cmm/CmmCallConv.hs cmm/CmmInfo.hs cmm/CmmLex.hs cmm/CmmLint.hs cmm/CmmLive.hs cmm/CmmOpt.hs cmm/CmmParse.hs cmm/CmmProcPoint.hs cmm/CmmUtils.hs cmm/Dataflow.hs cmm/MachOp.hs cmm/PprC.hs cmm/PprCmm.hs codeGen/Bitmap.hs codeGen/CgBindery.lhs codeGen/CgCallConv.hs codeGen/CgCase.lhs codeGen/CgClosure.lhs codeGen/CgCon.lhs codeGen/CgExpr.lhs codeGen/CgForeignCall.hs codeGen/CgHeapery.lhs codeGen/CgHpc.hs codeGen/CgInfoTbls.hs codeGen/CgLetNoEscape.lhs codeGen/CgMonad.lhs codeGen/CgParallel.hs codeGen/CgPrimOp.hs codeGen/CgProf.hs codeGen/CgStackery.lhs codeGen/CgTailCall.lhs codeGen/CgTicky.hs codeGen/CgUtils.hs codeGen/ClosureInfo.lhs codeGen/CodeGen.lhs codeGen/SMRep.lhs coreSyn/CoreFVs.lhs coreSyn/CoreLint.lhs coreSyn/CorePrep.lhs coreSyn/CoreSubst.lhs coreSyn/CoreSyn.lhs coreSyn/CoreTidy.lhs coreSyn/CoreUnfold.lhs coreSyn/CoreUtils.lhs coreSyn/ExternalCore.lhs coreSyn/MkExternalCore.lhs coreSyn/PprCore.lhs coreSyn/PprExternalCore.lhs cprAnalysis/CprAnalyse.lhs deSugar/Check.lhs deSugar/Coverage.lhs deSugar/Desugar.lhs deSugar/DsArrows.lhs deSugar/DsBinds.lhs deSugar/DsCCall.lhs deSugar/DsExpr.lhs deSugar/DsForeign.lhs deSugar/DsGRHSs.lhs deSugar/DsListComp.lhs deSugar/DsMeta.hs deSugar/DsMonad.lhs deSugar/DsUtils.lhs deSugar/Match.lhs deSugar/MatchCon.lhs deSugar/MatchLit.lhs ghci/ByteCodeAsm.lhs ghci/ByteCodeFFI.lhs ghci/ByteCodeGen.lhs ghci/ByteCodeInstr.lhs ghci/ByteCodeItbls.lhs ghci/ByteCodeLink.lhs ghci/Debugger.hs ghci/GhciMonad.hs ghci/GhciTags.hs ghci/InteractiveUI.hs ghci/Linker.lhs ghci/ObjLink.lhs ghci/RtClosureInspect.hs hsSyn/Convert.lhs hsSyn/HsBinds.lhs hsSyn/HsDecls.lhs hsSyn/HsDoc.hs hsSyn/HsExpr.lhs hsSyn/HsImpExp.lhs hsSyn/HsLit.lhs hsSyn/HsPat.lhs hsSyn/HsSyn.lhs hsSyn/HsTypes.lhs hsSyn/HsUtils.lhs iface/BinIface.hs iface/BuildTyCl.lhs iface/IfaceEnv.lhs iface/IfaceSyn.lhs iface/IfaceType.lhs iface/LoadIface.lhs iface/MkIface.lhs iface/TcIface.lhs main/BreakArray.hs main/CmdLineParser.hs main/CodeOutput.lhs main/Config.hs main/Constants.lhs main/DriverMkDepend.hs main/DriverPhases.hs main/DriverPipeline.hs main/DynFlags.hs main/ErrUtils.lhs main/Finder.lhs main/GHC.hs main/HeaderInfo.hs main/HscMain.lhs main/HscStats.lhs main/HscTypes.lhs main/InteractiveEval.hs main/Main.hs main/PackageConfig.hs main/Packages.lhs main/ParsePkgConf.hs main/PprTyThing.hs main/StaticFlags.hs main/SysTools.lhs main/TidyPgm.lhs nativeGen/AsmCodeGen.lhs nativeGen/GraphBase.hs nativeGen/GraphColor.hs nativeGen/GraphOps.hs nativeGen/GraphPpr.hs nativeGen/MachCodeGen.hs nativeGen/MachInstrs.hs nativeGen/MachRegs.lhs nativeGen/NCGMonad.hs nativeGen/PositionIndependentCode.hs nativeGen/PprMach.hs nativeGen/RegAllocColor.hs nativeGen/RegAllocInfo.hs nativeGen/RegAllocLinear.hs nativeGen/RegAllocStats.hs nativeGen/RegArchBase.hs nativeGen/RegArchX86.hs nativeGen/RegCoalesce.hs nativeGen/RegLiveness.hs nativeGen/RegSpill.hs nativeGen/RegSpillClean.hs nativeGen/RegSpillCost.hs ndpFlatten/FlattenInfo.hs ndpFlatten/FlattenMonad.hs ndpFlatten/Flattening.hs ndpFlatten/NDPCoreUtils.hs ndpFlatten/PArrAnal.hs parser/Ctype.lhs parser/HaddockLex.hs parser/HaddockParse.hs parser/HaddockUtils.hs parser/LexCore.hs parser/Lexer.hs parser/Parser.hs parser/ParserCore.hs parser/ParserCoreUtils.hs parser/RdrHsSyn.lhs prelude/ForeignCall.lhs prelude/PrelInfo.lhs prelude/PrelNames.lhs prelude/PrelRules.lhs prelude/PrimOp.lhs prelude/TysPrim.lhs prelude/TysWiredIn.lhs profiling/CostCentre.lhs profiling/SCCfinal.lhs rename/RnBinds.lhs rename/RnEnv.lhs rename/RnExpr.lhs rename/RnHsDoc.hs rename/RnHsSyn.lhs rename/RnNames.lhs rename/RnSource.lhs rename/RnTypes.lhs simplCore/CSE.lhs simplCore/FloatIn.lhs simplCore/FloatOut.lhs simplCore/LiberateCase.lhs simplCore/OccurAnal.lhs simplCore/SAT.lhs simplCore/SATMonad.lhs simplCore/SetLevels.lhs simplCore/SimplCore.lhs simplCore/SimplEnv.lhs simplCore/SimplMonad.lhs simplCore/SimplUtils.lhs simplCore/Simplify.lhs simplStg/SRT.lhs simplStg/SimplStg.lhs simplStg/StgStats.lhs specialise/Rules.lhs specialise/SpecConstr.lhs specialise/Specialise.lhs stgSyn/CoreToStg.lhs stgSyn/StgLint.lhs stgSyn/StgSyn.lhs stranal/DmdAnal.lhs stranal/SaAbsInt.lhs stranal/SaLib.lhs stranal/StrictAnal.lhs stranal/WorkWrap.lhs stranal/WwLib.lhs typecheck/FamInst.lhs typecheck/Inst.lhs typecheck/TcArrows.lhs typecheck/TcBinds.lhs typecheck/TcClassDcl.lhs typecheck/TcDefaults.lhs typecheck/TcDeriv.lhs typecheck/TcEnv.lhs typecheck/TcExpr.lhs typecheck/TcForeign.lhs typecheck/TcGadt.lhs typecheck/TcGenDeriv.lhs typecheck/TcHsSyn.lhs typecheck/TcHsType.lhs typecheck/TcInstDcls.lhs typecheck/TcMType.lhs typecheck/TcMatches.lhs typecheck/TcPat.lhs typecheck/TcRnDriver.lhs typecheck/TcRnMonad.lhs typecheck/TcRnTypes.lhs typecheck/TcRules.lhs typecheck/TcSimplify.lhs typecheck/TcSplice.lhs typecheck/TcTyClsDecls.lhs typecheck/TcTyDecls.lhs typecheck/TcTyFuns.lhs typecheck/TcType.lhs typecheck/TcUnify.lhs types/Class.lhs types/Coercion.lhs types/FamInstEnv.lhs types/FunDeps.lhs types/Generics.lhs types/InstEnv.lhs types/TyCon.lhs types/Type.lhs types/TypeRep.lhs types/Unify.lhs utils/Bag.lhs utils/Binary.hs utils/BufWrite.hs utils/Digraph.lhs utils/Encoding.hs utils/FastMutInt.lhs utils/FastString.lhs utils/FastTypes.lhs utils/FiniteMap.lhs utils/IOEnv.hs utils/ListSetOps.lhs utils/Maybes.lhs utils/OrdList.lhs utils/Outputable.lhs utils/Panic.lhs utils/Pretty.lhs utils/State.hs utils/StringBuffer.lhs utils/UniqFM.lhs utils/UniqSet.lhs utils/Util.lhs vectorise/VectBuiltIn.hs vectorise/VectCore.hs vectorise/VectMonad.hs vectorise/VectType.hs vectorise/VectUtils.hs vectorise/Vectorise.hs
.depend-BASE: openFile: resource busy (file is locked)
<<ghc: 475635736 bytes, 81 GCs, 4911104/12984320 avg/max bytes residency (5 samples), 27M in use, 0.00 INIT (0.00 elapsed), 0.86 MUT (7.97 elapsed), 0.14 GC (0.15 elapsed) :ghc>>
make[2]: *** [depend] Error 1
make[1]: *** [stage2] Error 2
make: *** [bootstrap2] Error 2

Warning: the following items did not execute (for ghc): org.macports.destroot org.macports.build
Error: Status 1 encountered during processing.

As I noted before, the locking bug showed up on Tiger, so it is very likely a ghc problem.

I am still puzzled by the fact that this bug doesn't seem to happen when I run make under bash in a terminal window. Building under Macports tickles the bug, though.

[simonmar]

Could you perhaps get a ktrace of the "file is locked" failure? I want to see which operation is generating the error. I took a quick look at the code and can't see anything obviously wrong.

[gw]

Once you see it, the bug is obvious. In rts/posix/FileLock.c, the lockFile routine only records a pointer from the first fd seen to the lock structure. If another fd read locks the same file, the second fd isn't recorded in the fd_hash, and the read lock count is not decremented when the second fd is closed.

Here is the patch that fixes the problem:

--- rts/posix/FileLock.c.sav	2008-02-20 05:47:14.000000000 -0500
+++ rts/posix/FileLock.c	2008-02-20 05:48:28.000000000 -0500
@@ -88,6 +88,12 @@
         if (for_writing || lock->readers < 0) {
             return -1;
         }
+
+	// have we seen this fd before?
+	if (lookupHashTable(fd_hash, fd) == NULL) {
+	    insertHashTable(fd_hash, fd, lock);
+	}
+
         lock->readers++;
         return 0;
     }

The last test in ghc-regress/lib/IO/openFile005 does put the hash tables into an inconsistent state, without tripping over the bug. I've added a test to trigger it.

I'll darcs send these as patches to the 6.8 branch.

The problem is not specific to OS X. It's still a bit puzzling why no one else has seen it, although as I mentioned before, it may have caused the version detection failure seen in Cabal.

[simonmar]

Aha! I already fixed this bug (#2122), and added a test for it. Thanks for diagnosing it, though.

I didn't add the "if (lookupHashTable(fd_hash, fd) == NULL)" test when I fixed it though, just unconditionally added the mapping to the hash table. Is that test necessary?

[gw]

Having looked at it again, the test "if (lookupHashTable(fd_hash, fd) == NULL)" is not probably not necessary, as long as we can guarantee that no fd will attempt to lock twice. A belt and suspenders approach is

--- /Users/gwright/src/macports-trunk/dports/lang/ghc/work/ghc-6.8.2/rts/posix/FileLock.c.sav	2008-03-03 11:40:14.000000000 -0500
+++ /Users/gwright/src/macports-trunk/dports/lang/ghc/work/ghc-6.8.2/rts/posix/FileLock.c	2008-03-03 11:43:40.000000000 -0500
@@ -88,6 +88,17 @@
         if (for_writing || lock->readers < 0) {
             return -1;
         }
+
+	// have we seen this fd before?
+	if (lookupHashTable(fd_hash, fd) == NULL) {
+		insertHashTable(fd_hash, fd);
+	}
+	else
+	{
+		errorBelch("fd %d tries to lock the same file twice", fd);
+		return -1;
+	}
+
         lock->readers++;
         return 0;
     }

I'm using this in the new macports 6.8.2 until I can convince myself that double locking is really impossible.

[gw]

Of course, what I meant was

--- rts/posix/FileLock.c.sav	2008-03-03 11:40:14.000000000 -0500
+++ rts/posix/FileLock.c	2008-03-03 11:43:40.000000000 -0500
@@ -88,6 +88,17 @@
         if (for_writing || lock->readers < 0) {
             return -1;
         }
+
+	// have we seen this fd before?
+	if (lookupHashTable(fd_hash, fd) == NULL) {
+		insertHashTable(fd_hash, fd, lock);
+	}
+	else
+	{
+		errorBelch("fd %d tries to lock the same file twice", fd);
+		return -1;
+	}
+
         lock->readers++;
         return 0;
     }