[Genomedata-users] PPC Mac failure on test the
PyTables Installation
Orion Buske
stasis at u.washington.edu
Wed May 26 10:03:23 PDT 2010
Hmm. It doesn't look like the new prints are in there. Sorry about that.
I still don't know what the issue was, but I don't think we have ever
tested it on PPC Macs, so there may be something peculiar about it.
Anyway, sorry for never working it out.
If you don't mind me asking, what issues did you run into on Ubuntu? We
are very interested in making it as easy to install as we can.
Thanks! I hope you find Genomedata useful,
Orion
David Mead wrote:
> Hi Orion,
>
> I ended up trying the install on Ubuntu 9.10. There were a few issues,
> but I was able to get things up and running, so I am giving up on the
> Mac for now.
>
> I did try the latest code you gave below, but the output did not appear
> to have any new debug statements in it. If you would like me to do any
> more testing on the Mac, let me know.
>
> Thank you for all your help,
>
> David Mead
>
> Output from last run:
>
> david-meads-power-mac-g5:test dwm08$ genomedata-load -f -v -s
> data/chr1.short.fa -s data/chrY.short.fa.gz -t
> placental=data/chr1.phyloP44way.placental.short.wigFix -t
> primate=data/chr1.phyloP44way.primate.short.wigFix test.genomedata
> >> Using temporary Genomedata archive: /tmp/tmp3o8Fw1.genomedata
> >> Loading sequence files:
> data/chr1.short.fa
> data/chrY.short.fa.gz
> >> Opening Genomedata archive with 2 tracks
> >> Loading data for track: placental
> chr1
> allocating data buffer of size:
> 1986068480x4...genomedata-load-data(65662) malloc: ***
> mmap(size=3649306624) failed (error code=12)
> *** error: can't allocate region
> *** set a breakpoint in malloc_error_break to debug
> virtual memory exhausted Error loading data from track file:
> data/chr1.phyloP44way.placental.short.wigFix
> Error creating genomedata!
> >> Cleaning up... done
>
>
> On 05/24/2010 12:39 PM, Orion Buske wrote:
>> David,
>>
>> I honestly don't know why the size of the buffer it is trying to
>> allocate is so large on your machine. I can't reproduce this behavior.
>> My gut thinks something funny is going on with unsigned ints or longs
>> or something like that. All I did was add a few more diagnostic prints
>> to see if we can track down what's going on. I honestly don't see
>> anything that jumps out, but hopefully this will give us something to
>> work with.
>>
>> Same drill:
>> http://noble.gs.washington.edu/~stasis/outbox/david_mead/genomedata-1.1.2.dev-r3273.tar.gz
>>
>>
>>
>> Sorry this didn't have a more straightforward solution,
>> Orion
>>
>> David Mead wrote:
>>> Orion,
>>>
>>> Mac doesn't have meminfo, but I got this from top:
>>>
>>> Processes: 58 total, 3 running, 55 sleeping... 220
>>> threads
>>> 10:07:36
>>> Load Avg: 0.01, 0.01, 0.00 CPU usage: 1.30% user, 6.09% sys,
>>> 92.61% idle
>>> SharedLibs: num = 2, resident = 85M code, 0 data, 5632K
>>> linkedit.
>>> MemRegions: num = 7011, resident = 237M + 19M private, 111M shared.
>>> PhysMem: 256M wired, 469M active, 183M inactive, 943M used, 1617M
>>> free.
>>> VM: 4889M + 139M 50134(0) pageins, 0(0) pageouts
>>>
>>> I compiled the latest and ran it. Here is what I got.
>>>
>>> david-meads-power-mac-g5:test dwm08$ genomedata-load -f -v -s
>>> data/chr1.short.fa -s data/chrY.short.fa.gz -t
>>> placental=data/chr1.phyloP44way.placental.short.wigFix -t
>>> primate=data/chr1.phyloP44way.primate.short.wigFix test.genomedata
>>> >> Using temporary Genomedata archive: /tmp/tmpmLKuze.genomedata
>>> >> Loading sequence files:
>>> data/chr1.short.fa
>>> data/chrY.short.fa.gz
>>> >> Opening Genomedata archive with 2 tracks
>>> >> Loading data for track: placental
>>> chr1
>>> allocating data buffer of size:
>>> 1986068480x4...genomedata-load-data(51610) malloc: ***
>>> mmap(size=3649306624) failed (error code=12)
>>> *** error: can't allocate region
>>> *** set a breakpoint in malloc_error_break to debug
>>> virtual memory exhausted Error loading data from track file:
>>> data/chr1.phyloP44way.placental.short.wigFix
>>> Error creating genomedata!
>>> >> Cleaning up... done
>>> david-meads-power-mac-g5:test dwm08$
>>>
>>>
>>>
>>>
>>> On 05/21/2010 06:34 PM, Orion Buske wrote:
>>>> Okay David,
>>>>
>>>> There appears to be something wrong with how much memory is being
>>>> allocated. I added a debug print to the allocation step and packaged
>>>> up the latest HEAD version of Genomedata for you. We're going to
>>>> install this over the previous version and see 1) if it fixes the
>>>> problem (unlikely) or 2) how much memory you're allocating.
>>>>
>>>> $ wget
>>>> http://noble.gs.washington.edu/~stasis/outbox/david_mead/genomedata-1.1.2.dev-r3273.tar.gz
>>>>
>>>> $ tar -xzf genomedata-1.1.2.dev-r3273.tar.gz
>>>> $ cd genomedata-1.1.2.dev-r3273
>>>> $ python setup.py install
>>>>
>>>> And then run that test command.
>>>>
>>>> Could you also send the output of the following command:
>>>> $ cat /proc/meminfo
>>>> (or whatever the Mac equivalent is?)
>>>>
>>>> Thanks,
>>>> Orion
>>>>
>>>> David Mead wrote:
>>>>> Hi Orion,
>>>>>
>>>>> I think we made progress, but still not quite there. I tried the
>>>>> test again and got this:
>>>>>
>>>>> genomedata-load -f -v -s data/chr1.short.fa -s
>>>>> data/chrY.short.fa.gz -t
>>>>> placental=data/chr1.phyloP44way.placental.short.wigFix -t
>>>>> primate=data/chr1.phyloP44way.primate.short.wigFix test.genomedata
>>>>> >> Using temporary Genomedata archive: /tmp/tmpKu0R5D.genomedata
>>>>> >> Loading sequence files:
>>>>> data/chr1.short.fa
>>>>> data/chrY.short.fa.gz
>>>>> >> Opening Genomedata archive with 2 tracks
>>>>> >> Loading data for track: placental
>>>>> chr1
>>>>> genomedata-load-data(38245) malloc: *** mmap(size=3649306624)
>>>>> failed (error code=12)
>>>>> *** error: can't allocate region
>>>>> *** set a breakpoint in malloc_error_break to debug
>>>>> virtual memory exhausted Error loading data from track file:
>>>>> data/chr1.phyloP44way.placental.short.wigFix
>>>>> Error creating genomedata!
>>>>> >> Cleaning up... done
>>>>>
>>>>> Thanks,
>>>>>
>>>>> David
>>>>>
>>>>> On 05/21/2010 05:19 PM, Orion Buske wrote:
>>>>>> Hi David,
>>>>>>
>>>>>> I believe that genomedata_load_data.c is getting compiled before
>>>>>> the egg is created, thus making the "easy_install genomedata" grab
>>>>>> a pre-compiled genomedata-load-data instead of compiling it during
>>>>>> installation. The fix is as follows:
>>>>>>
>>>>>> 1) Download the source:
>>>>>> $ wget
>>>>>> http://noble.gs.washington.edu/proj/genomedata/genomedata-1.1.1.tar.gz
>>>>>>
>>>>>>
>>>>>> 2) Install it:
>>>>>> $ tar -xzf genomedata-1.1.1.tar.gz
>>>>>> $ cd genomedata-1.1.1
>>>>>> $ python setup.py install
>>>>>>
>>>>>>
>>>>>> Then you can try that genomedata-load command from before to test,
>>>>>> but this should work.
>>>>>>
>>>>>> Cheers,
>>>>>> Orion
>>>>>>
>>>>>> David Mead wrote:
>>>>>>> Looks like genomedata-load-data does have problems. I've also
>>>>>>> attached the original install log in case that can give a clue as
>>>>>>> to what went wrong.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> david-meads-power-mac-g5:~ dwm08$ which genomedata-load-data
>>>>>>> /Users/dwm08/arch/Darwin-Power/bin/genomedata-load-data
>>>>>>> david-meads-power-mac-g5:~ dwm08$ genomedata-load-data
>>>>>>> -bash: /Users/dwm08/arch/Darwin-Power/bin/genomedata-load-data:
>>>>>>> cannot execute binary file
>>>>>>> david-meads-power-mac-g5:~ dwm08$ easy_install genomedata
>>>>>>> Searching for genomedata
>>>>>>> Best match: genomedata 1.1.1
>>>>>>> Processing genomedata-1.1.1-py2.5.egg
>>>>>>> genomedata 1.1.1 is already the active version in easy-install.pth
>>>>>>> Installing genomedata-load-data script to
>>>>>>> /Users/dwm08/arch/Darwin-Power/bin
>>>>>>> Installing genomedata-load-seq script to
>>>>>>> /Users/dwm08/arch/Darwin-Power/bin
>>>>>>> Installing genomedata-open-data script to
>>>>>>> /Users/dwm08/arch/Darwin-Power/bin
>>>>>>> Installing genomedata-load script to
>>>>>>> /Users/dwm08/arch/Darwin-Power/bin
>>>>>>> Installing genomedata-close-data script to
>>>>>>> /Users/dwm08/arch/Darwin-Power/bin
>>>>>>> Installing genomedata-report script to
>>>>>>> /Users/dwm08/arch/Darwin-Power/bin
>>>>>>> Installing genomedata-erase-data script to
>>>>>>> /Users/dwm08/arch/Darwin-Power/bin
>>>>>>>
>>>>>>> Using
>>>>>>> /Users/dwm08/arch/Darwin-Power/lib/python2.5/genomedata-1.1.1-py2.5.egg
>>>>>>>
>>>>>>> Processing dependencies for genomedata
>>>>>>> Finished processing dependencies for genomedata
>>>>>>>
More information about the Genomedata-users
mailing list