Andy Turner's Blog 2006-09 Page

Andy Turner's Blog

2006-10-01

2006-09-29

• PhD meeting with Andy and Mark
• MoSeS
  • Colin Venters phoned to set up a collaboration with his new projecton ADVISES - (visualisation for epidemiology). We should collaborate :)
  • loads of coding...

2006-09-28

• MoSeS
  • Major code tidy and name standardisation of names in source code and documentation
    • IndividualSAR is now ISAR and HouseholdSAR is HSAR

2006-09-27

• Diane has asked me to be a tutor for two students from module 1240... I have confirmed that I am willing and await further instructions...
• MoSeS
  • Found a logic error in MoSeS code!
    • This was found thanks to Mark's insistance to analyse the results :)
    • The error crept in when changes were made to constrain by HRP ages only.
    • In initialising a result the eldest people are assigned as HRP
    • When optimising a set of HRPs are swapped only with another set of HRPs is fair enough, When optimising
    • More processing woes! Is it MPJExpress that is behaving badly... Emailed MPJ user list to enquire about the limitation of Barrier()
    • Comparing results from OA_HSARHP_ISARCEP with CASUV003 there are some large differences. Is this due to SCAM or some logic error in MoSeS code?
  • Filed expenses claim for AHM2006
• Communications with Jennifer Forde of the BBC researching into commuting migration and motorway use since the 1960s.
  • Emailed Oli to see if he is interested...

2006-09-26

• Meeting with Mark Birkin and Stuart Roberts who is doing a JISC survey on Geospatial DataSets
  • Outlined MoSeS and the data we are interested in and add value to.
  • Discussed DDI and provenance.
  • Discussed Data Integration.
  • Steve Carver is organising a meta data workshop in November.
  • Suggested Stuart contact cluster leaders in SoG to ask what primary data they collect.
  • Actions
    • Andy to email Stuart a link to our MoSeS work online
• Meeting with Mark about MoSeS and progress with data cooking.
  • Time to focus on presenting results that have been generated...
  • Prepare for meeting on 2006-10-04
• Filed expenses claim for FOSS4G2006

2006-09-25

• Beowulf processing failed again. Not sure what is going wrong, but keep getting stuck at the same stage...
• Javascript to manage Web Content
  • Mozilla Core_JavaScript_1.5_Guide
  • Mozilla Core_JavaScript_1.5_Reference
  • http://developer.mozilla.org/en/docs/document
  • http://developer.mozilla.org/en/docs/Gecko_DOM_Reference
  • http://www.w3.org/DOM/
  • http://www.w3.org/TR/DOM-Level-3-Core/
  • ECMAScript Language Binding
  • http://www.w3schools.com/htmldom/

2006-09-22

• Committed to a Joint Meeting of the GEMS/GEESE/GRID OGC Collision - October 10th, 2006 - Jurys Inn Birmingham
• Initialised My DSOnline Gaming And Simulation Web Page.
• Informal meeting with Mark to catch up.
• I've got masses of catching up to do!
• The Beowulf machine seems to be broken. I've emailed Mike to look into it...
• James Farnhill from JISC who I met at AllHands put me on to http://www.rss-to-javascript.com/p/138.html
  • We want to convert our RSS feeds into HTML pages using javascript
  • The hope is to have an alternative to Server Side Includes so that Web Content is more manageable, i.e. there is less duplication of code.
  • Test 1.
    • The javascript calls for a service provided by RSS-to-JavaScript.com
      • The page takes a little time to load. What I think happens is that the javascript invokes the service to convert the rdf document, the service takes the RDF and produces some html via PhP and this is then injected into the html.
      • Nice :)
      • Although nice, I wonder how reliable this service is and weather it really meets my requirements.
      • I'll be giving this some more thought.
      • It is my understanding that I need to put javascript in a sperate file and link to it in the header of the XHTML2.0.
      • I wonder how to do this and achieve the same effect...

2006-09-18

• Back from FOSS4G2006...
• MoSeS
  • Emailed Mike Crabtree to ask him to open a port for a tomcat server on the MoSeS portal machine so it can communicate with the outside world.
  • Poster for CSAP launch
    • Mark has provided Helen with a poster pdf
    • I am to organise media services production of an A1 poster from this.
  • Processing failed...
    • Exception on 1 with MPI.COMM_WORLD.Send to 0
    • No idea yet why this happened.
    • Here is the error report:
      • # # An unexpected error has been detected by HotSpot Virtual Machine: # # SIGSEGV (0xb) at pc=0x402c24ea, pid=18108, tid=1098087616 # # Java VM: Java HotSpot(TM) Client VM (1.5.0_05-b05 mixed mode) # Problematic frame: # V [libjvm.so+0x2944ea] # # An error report file with more information is saved as hs_err_pid18108.log # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp EXITmpi.MPIException: xdev.XDevException: java.nio.channels.ClosedChannelException at mpi.Comm.send(Comm.java:378) at mpi.Comm.Send(Comm.java:329) at uk.ac.leeds.ccg.andyt.projects.moses.process.IndividualCensusBeowulf.run_MSOA_HSARHP_ISARCEP(IndividualCensusBeowulf.java:950) at uk.ac.leeds.ccg.andyt.projects.moses.process.IndividualCensusBeowulf.run(IndividualCensusBeowulf.java:130) at uk.ac.leeds.ccg.andyt.projects.moses.process.IndividualCensusBeowulf.main(IndividualCensusBeowulf.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at runtime.daemon.JarClassLoader.invokeClass(JarClassLoader.java:111) at runtime.daemon.ThreadedWrapper$1.run(ThreadedWrapper.java:235) at java.lang.Thread.run(Thread.java:595) Caused by: xdev.XDevException: java.nio.channels.ClosedChannelException at xdev.niodev.NIODevice.isend(NIODevice.java:1404) at xdev.niodev.NIODevice.send(NIODevice.java:1563) at mpjdev.Comm.send(Comm.java:897) at mpi.Comm.send(Comm.java:370) ... 11 more Caused by: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:125) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:294) at xdev.niodev.NIODevice.rendezCtrlMsgSend(NIODevice.java:1631) at xdev.niodev.NIODevice.isend(NIODevice.java:1400) ... 14 more mpi.MPIException: xdev.XDevException: java.nio.channels.ClosedChannelException at mpi.Comm.send(Comm.java:378) at mpi.Comm.Send(Comm.java:329) at uk.ac.leeds.ccg.andyt.projects.moses.process.IndividualCensusBeowulf.run_MSOA_HSARHP_ISARCEP(IndividualCensusBeowulf.java:950) at uk.ac.leeds.ccg.andyt.projects.moses.process.IndividualCensusBeowulf.run(IndividualCensusBeowulf.java:130) at uk.ac.leeds.ccg.andyt.projects.moses.process.IndividualCensusBeowulf.main(IndividualCensusBeowulf.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at runtime.daemon.JarClassLoader.invokeClass(JarClassLoader.java:111) at runtime.daemon.ThreadedWrapper$1.run(ThreadedWrapper.java:235) at java.lang.Thread.run(Thread.java:595) Caused by: xdev.XDevException: java.nio.channels.ClosedChannelException at xdev.niodev.NIODevice.isend(NIODevice.java:1404) at xdev.niodev.NIODevice.send(NIODevice.java:1563) at mpjdev.Comm.send(Comm.java:897) at mpi.Comm.send(Comm.java:370) ... 11 more Caused by: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:125) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:294) at xdev.niodev.NIODevice.rendezCtrlMsgSend(NIODevice.java:1631) at xdev.niodev.NIODevice.isend(NIODevice.java:1400) ... 14 more mpi.MPIException: xdev.XDevException: java.nio.channels.ClosedChannelException at mpi.Comm.send(Comm.java:378) at mpi.Comm.Send(Comm.java:329) at uk.ac.leeds.ccg.andyt.projects.moses.process.IndividualCensusBeowulf.run_MSOA_HSARHP_ISARCEP(IndividualCensusBeowulf.java:950) at uk.ac.leeds.ccg.andyt.projects.moses.process.IndividualCensusBeowulf.run(IndividualCensusBeowulf.java:130) at uk.ac.leeds.ccg.andyt.projects.moses.process.IndividualCensusBeowulf.main(IndividualCensusBeowulf.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at runtime.daemon.JarClassLoader.invokeClass(JarClassLoader.java:111) at runtime.daemon.ThreadedWrapper$1.run(ThreadedWrapper.java:235) at java.lang.Thread.run(Thread.java:595) Caused by: xdev.XDevException: java.nio.channels.ClosedChannelException at xdev.niodev.NIODevice.isend(NIODevice.java:1404) at xdev.niodev.NIODevice.send(NIODevice.java:1563) at mpjdev.Comm.send(Comm.java:897) at mpi.Comm.send(Comm.java:370) ... 11 more Caused by: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:125) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:294) at xdev.niodev.NIODevice.rendezCtrlMsgSend(NIODevice.java:1631) at xdev.niodev.NIODevice.isend(NIODevice.java:1400) ... 14 more
    • Sussed it :) Phew! A synopsis follows
      • Problem: The process ran to completion only iterating through an optimisation a small number of times but failed when iterating through an optimisation a large number of times.<>
      • Cause: The master had farmed out a set of tasks, one processor had finished and recieved another task and finished this before one of the previos tasks had completed.
      • Solution: Barriers were put back in place to ensure that all processes finished the first task before starting the second. This slows things down, but is more robust.
      • A better solution is perhaps to have each slave process write out results to a different file and to append these at the end.
• Need to prepare for All Hands
  • UK-China Workshop

2006-09-08 - 2006-09-15

• FOSS4G2006

2006-09-07

• MoSeS
  • Looks like I was right about sampling without replacement and the size of the Household SAR
  • Next I suspect I will be asked to produce sampling with replacement versions
    • That shouldn't be too difficult...
    • Should have done it from the start... It would have been much easier than what was done!
• Chat with Nik and Paul about AllHands UK-China collaboration workshop Key Note :)
• Completed preperations for FOSS4G2006

2006-09-06

• Preparation for FOSS4G2006
• Paul came and gave me some moral support. Thanks Paul :)

2006-09-05

• Still not feeling up to doing anything difficult after weekend over excess!
• MoSeS processing runs are still getting stuck for some reason... I'm starting to wonder if it's hardware or resource problems...

2006-09-04

• It's all gone pair shaped! Can't really remember what I was doing. Overdid it at the weekend nightmare!
• Wrote a list... Tried to work through it... Couldn't do anything... Need to recover and get my head in gear...
• Felt much better after a game of squash and a hearty meal :)

2006-09-01

• All Hands
  • UK-China Workshop
    • Got an email from Nik
    • I need to organise a 20 minute talk hopefully on MOSES, but talking about how you could possibly make use of the Grid infrastructures available in China (possibly through contacts made through the CoLaB collaboration) to process data etc.
    • Need to find out more about CoLaB...
• MoSeS
  • Documentation
    • Updated MoSeS 2001 UK Demographic Initialisation Page
  • Data Processing
    • Third time lucky... In tidying the methods I created a NullPointerException bug when results were to be written out...
  • Programming
    • Removed NullPointerException bug.
    • Improved random data seeding so that it should be easy to replicate results in parallel processing. Before this was not the case as all processes were sharing the same Random...

2006-08-31