Andy Turner's Blog 2009-10 Web Page @ School of Geography, University of Leeds

Blog
Events:
- NGS Innovation Forum '09
  - http://www.ngs.ac.uk/innovationforum09
  - 2009-10-21 to 2009-10-22, London, UK
- Free and Open Source Software for Geospatial (FOSS4G 2009)
  - http://2009.foss4g.org/
  - 2009-10-20 to 2009-10-23, Sydney, Australia.
- Workshop on The impact and influence of Web 2.0 on e-Research
  - http://research3.org/events/workshop_two.php
  - 2009-10-13, Banff, Canada.
  - Held in conjunction with OGF27 2009-10-12 to 2009-10-16
- OGC Technical and Planning Committee Meetings September '09
  - http://www.opengeospatial.org/event/0909tc
  - 2009-09-28 to 2009-10-02, Darmstadt, Germany
- Mapping Information with and without Geography: Approaches to Data Visualization and Structure in the Arts, Humanities and Social Sciences
  - http://www.nesc.ac.uk/esi/events/1005/
  - 2009-09-30 to 2009-10-01, Edinburgh, UK.
- Events on the horizon:
  - GIScience 2010
    - http://www.giscience2010.org/
    - 2010-09-14 to 2010-09-17, Zurich, Switzerland.
  - Workshop on Urban Dynamics URBANICS
    - http://www.urbanics.cl
    - 2010-03-22 to 2010-03-25, Chile
  - International Symposium on Grid Computing (ISGC) 2010
    - http://event.twgrid.org/isgc2010/
    - 2010-03-07 to 2010-03-10, Taipei, Taiwan.
  - The 2nd International Conference on Computer modeling and simulation (ICCMS 2010)
    - http://www.iccms.org
    - 2010-01-22 2010-01-24, Sanya, China.
  - UK e-Science AHM 2009
    - http://www.allhands.org.uk/
    - 2009-12-07 to 2009-12-09, Oxford, UK.
  - ComputationWorld 2009
    - http://www.iaria.org/conferences2009/ComputationWorld09.html
    - 2009-11-15 to 2009-11-20, Athens, Greece.
2009-11-02
2009-10-28
- Browsing
- e-Science
  - OneVRE meeting
    - http://wiki.rcs.manchester.ac.uk/community/OneVRE
    - People
      - Paul Lambert
      - Mike Poschen
      - Tobias Schiebeck
      - Andy Turner
      - Martin Turner
      - Tim Parkinson
      - ...
    - Presentation
      - http://192.150.184.103:7890/
    - Meik Poschen is to circulate minutes...
    - PAG working with Sakai
    - Browsing
    - Link to a PAG client that I can use:
      - http://pag.rcs.manchester.ac.uk
  - GENESIS
    - Developing Agent and AgentCollection and AgentCollectionManagement code
      - ...
2009-10-27
- Browsing
  - http://www.gvsigmini.org
- Miscellanea
  - CSAP research cluster meeting
    - Business
      - Justin Hayes plans to visit to talk about the InFuse Project
        
        http://cdu.mimas.ac.uk/projects/infuse.html
        
        To help researchers, teachers and students to understand, explore, obtain and use UK census information in innovative ways, by developing a new online application to exploit recent advances in dissemination methods and technologies - and to expand and enhance the use of this important socio-economic resource.
        
        2009-12-21
      - WUN Seminar 5pm tomorrow on Epidemiological modelling
    - Presentation on Integrating Population Data
      - Kirk Harland
      - Work with Alison Heppenstall to use Acxium (http://en.wikipedia.org/wiki/Acxiom) data to assign simulated individuals behaviour
      - Compared:
        
        Deterministic reweighting
        
        Conditional Probablility
        
        Combinatorial Optimisation using simulated annealing
      - Concluded that 3 is best and 1 is worst.
      - Kirk has been doing similar work to what I did on MoSeS, but also factoring in Small Area Microdata (SAM) - a 5% sample for the UK
      - Limiting Long Term Illness (LLTI) is a tricky constraint
        
        I suggested this might be because a relatively small proportion of population are LLTI and so using this as a constraint quickly narrows the sample...
      - For Leeds, the Harehills area is an outlier in ethnicity
        
        I wonder if my results are the same...
      - Afterwards I sat with Kirk and showed him some MoSeS results:
        
        http://www.geog.leeds.ac.uk/people/a.turner/projects/MoSeS/documentation/demography/results/2001PopulationInitialisation/ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_500_10_100_12_10/Leeds/ControlConstraints/OA.xhtml2.0.html
        
        http://www.geog.leeds.ac.uk/people/a.turner/projects/MoSeS/documentation/demography/results/2001PopulationInitialisation/ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_500_10_100_12_10/Leeds/OptimisationConstraints/OA.xhtml2.0.html
        
        http://www.geog.leeds.ac.uk/people/a.turner/projects/MoSeS/documentation/demography/results/2001PopulationInitialisation/ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_500_10_100_12_10/Leeds/NonConstraints/OA.xhtml2.0.html
      - We thought it might be worth generating comparable results and doing a comparison...
      - I asked Kirk to open source his software and make it available...
- e-Science
  - Meeting with Mark Birkin about what to ask of Jin
    - Demonstration videos and documentation for installing and running...
    - Univariate table harmonisation for the UK
    - Delivery of the UK System
    - Development of Archive and metadata
2009-10-26
- Browsing
- Miscellanea
  - PC disk corruption
    - Failure to boot.
    - Disk mounted on another machine and a File System Check seems to have resolved this issue.
  - MASS weekly catch up meeting
    - I mentioned the work I am doing scaling up GENESIS models, in particular, the File Cache implementation that I was working on last week and will be testing further this week...
    - ...
2009-10-23
- e-Science
  - GENESIS
    - Developing a FileStore for agent modelling
      - ...
2009-10-22
- Browsing
- Miscellanea
  - Read Sun Microsystem White Paper - Open Virtualization
- Teaching
  - GEOG1300 Group Tutorial Meeting
    - We spent some time reflecting on the course...
    - I feel like I should do the reading for this module and I am concerned about my lack of practice in writing essays.
- e-Science
  - e-ISS
    - Registered http://www.neiss.org.uk on http://en.wikipedia.org/wiki/Social_simulation
      - The National e-Infrastructure for Social Simulation (NeISS) is a UK based project funded by [http://www.jisc.ac.uk JISC]. For further details please see: [http://www.neiss.org.uk The NeISS Web Pages].
  - GENESIS
    - Registered http://www.genesis.ucl.ac.uk/ and http://www.casa.ucl.ac.uk/genesisblog/ on http://en.wikipedia.org/wiki/Social_simulation
      - Generative e-Social Science for Socio-Spatial Simulation (GENESIS) is a research node of the UK National Centre for e-Social Science funded by the UK [http://www.esrc.ac.uk ESRC]. For further details please see: [http://www.genesis.ucl.ac.uk/ GENESIS Web Page] and [http://www.casa.ucl.ac.uk/genesisblog/ Blog].
2009-10-21
- Browsing
- Miscellanea
  - Chat with Bobbie Millar about some work she is hoping to do with the National Trust on the Fountains Abbey estate
    - The main idea is to develop a load of 3D type models of the site to show its state past, present and under alternative futures for different interventions.
    - A project was planned that involved historians, archeologists and experts in river diversion (including Joe Holden) and others, but it did not get funded initially.
    - Clearly this sort of thing can be done for other sites too and so this could lead on to more work...
    - She was hoping I knew of someone with skills in using Autodesks Maya software:
      - http://www.autodesk.co.uk/adsk/servlet/pc/index?id=14595079&siteID=452932
    - I suggested she ask Andy Hudson-Smith at CASA UCL and others via the Digital Urban Blog.
- e-Science
  - ...
  - e-ISS
    - http://portal.ncess.ac.uk/access/wiki/site/e-iss/2009-11-04.html
    - http://portal.ncess.ac.uk/access/wiki/site/e-iss/2009-10-21.html
2009-10-20
- Browsing
  - ...
- Teaching
  - GEOG1300
    - Meeting with Paul Waley
      - Assessment, documentation and finding out what modules students are doing
        
        ...
- e-Science
  - ...
2009-10-19
- Browsing
  - http://www.twinity.com
  - http://www.genesis.ncess.ac.uk/
- e-Science
  - ...
2009-10-16
- Miscellanea
  - My work laptop was stolen :-(
    - This is a considerable set back... I have to do various things because of the data/certificates that were on it... I think I'm going to wish the whole thing were encrypted... Can I cope without a laptop? Well, I feel like I need a laptop for my work these days, but I will manage for as long as I have to... Getting a new one configured with all the software I use will take time...
- e-Science
  - e-ISS
    - Meeting with Taverna developers Shoaib Sufi and Alex Nenadic
      - Agenda
        
        GENESIS code compiling and running.
        
        Discuss/brainstorm
        
        Leeds based activities
        
        Issues
        
        How to collaborate with regards Taverna/NeISS...
      - Notes
        
        Andy could not demo some things as he had hoped as his laptop was stolen earlier in the day :-(
        
        GENESIS code compiling and running
        
        We got Andy's GENESIS code compiling and successfully ran one process on Alex' laptop.
        
        It would be better if Andy did releases with all dependent libraries made available too.
        
        The only process that ran without error was a simple demographic example based on syntheitc program generated data.
        
        Andy thinks he knows the problem with other processes and will fix this and ask Alex to update...
        
        Discuss/brainstorm
        
        Leeds based activities
        
        Other key workers based at Leeds for this are Paul Townend and Jianhui Jin
        
        Jin has developed a population initialisation (Population Reconstruction Model) software that can be actioned via a servlet and has been tested with LifeRay
        
        Unfortunately this uses census data for which getting a licence is not straightforward.
        
        Andy had hoped to demo this...
        
        Paul has worked with Taverna and has developed a set of MoSeS and now GENESIS portlets that enact web services in a form of work flow.
        
        File based storage of data inputs and outputs
        
        ...
        
        Issues
        
        How to collaborate with regards Taverna/NeISS...
  - NeSS
    - Meeting with Rob Procter, June Finch and Pascal Ekin
      - Agenda
        
        Demonstrators
        
        Collaboration with EUAsiaGrid
      - Notes
        
        Demonstrators
        
        What to do with regard developing demonstrators
        
        What form can demonstrators take?
        
        They could be movies of a working system and documentation to assist with setting such a system up rather than a working tool that any user can potentailly access and use.
        
        What components are wanted
        
        Ideally each demonstrator will draw on all aspects of the project including security and metadata registries etc...
        
        Think of a workflow use case
        
        Select region
        
        Select spatial output boundaries
        
        Specify model
        
        Use existing output, or generate new results from source
        
        Algorithm parameters
        
        Variable selection
        
        Detail optimisation function
        
        Select resource to run on
        
        Run
        
        Get result
        
        Visualise result
        
        Archive result
        
        Implemented well, this would be a population initialisation/reconstruction tool.
        
        I demonstrated Jin's progress on his population initialisation/reconstruction tool for NeISS...
        
        I demonstrated the PGrade interface to the initialisation/reconstruction I developed with the help of colleagues at the University of Westminster for MoSeS/EUAsiaGrid...
        
        We talked about ConverterGrid/ConvertGrid.
        
        We talked about different ways of accessing 2001 census aggregate statistics
        
        Via CASWEB and ONS
        
        Rob mentioned Felix Ritchie from ONS as somone who might be able to help
        
        We both participated in a NDS-NCeSS Workshop on Secure Access to Confidential Data
        
        http://portal.ncess.ac.uk/access/wiki/site/~a.g.d.turner@leeds.ac.uk/2008-06-26.html
2009-10-15
- Browsing
- Miscellanea
  - Registered Grids on OSGEO_Labs
  - MASS Meeting
    - Kirk showed an interface to some software he has been writing called a Microsimulation Modelling Framework
    - We had some good discussion about work others had been doing to do with selecting from sample surveys to comprise populations of regions.
    - I suggested we work up some sythetic data, including a 100% annonymised record set, so we could run some tests.
- e-Science
  - NeSS
    - Brief meeting with Paul
      - He is confident about configuration/migration of MoSeS Portlets in the NCeSS Sakai Portal.
  - GENESIS
    - Project Meeting
      - Unfortunately I lost the notes I took during the meeting and this is written from memory and only a relatively brief summary
      - People
        
        Mark Birkin
        
        Alison Heppenstall
        
        Paul Townend
        
        Andy Turner
        
        Belinda Wu
      - Round table on progress
        
        Paul showed the Web Service Based Version 0.3.8 of the GENESIS Portlets he has been developing
        
        It was working in LifeRay and looked good.
        
        IMO Paul seems to be overly fussing about presentation, but still presentation is important...
        
        Paul planning to move from using Axis 1 to 2 allowing for asynchronous communication which should help with the user experience and not tie up the interface waiting for a web service to return or not.
        
        Andy demonstrated progress on developing traffic and a demographic models, and the system Jianhui Jin has been developing for NeISS
        
        Belinda has been working on the fertility and mortality aspects of her dynamic demographic model.
        
        Mark is embarking upon a campaign of events to advertise what we are doing :-)
        
        Alison has been working with Kirk to develop a Microsimulation Modelling Framework Software and on publications comparing the outputs of different algorithms for sampling from a sample population to represent the population of a region.
      - Paul pointed me to http://www.gapminder.org/
      - Chris Thompson
      - Nick Malleson starts work on NeISS on 2010-02-01
      - Geoff Hewings
        
        http://en.wikipedia.org/wiki/Geoffrey_J.D._Hewings
      - Luke Burns
      - Andrew Crooks
      - Dave DeRoure
        
        New head of NCeSS
      - Next meeting date proposed 2009-10-19
    - Scaling up to the UK
      - The UK human population is approximately 60 million.
      - I have figured a way of storing information in a file system for these 60 million representative entities.
      - However iterating over this collection is reasonably slow where I use 1 file for each representative entity.
      - One experiment I did was:
        
        To create a directory structure containing nested directories of 100.
        
        At the leaves of the tree were directories for each individual represented.
        
        The directory structure was 5 directories deep and easily navigable.
        
        I iterated over 1 million entities in about 100 minutes.
      - Now I am thinking of grouping representative entities into collections of say 1000, and swapping these from files to memory
        
        I'm confident this will speed things up, but I also expect to need increasingly sophisticated memory handling.
        
        It is probably best to keep the file to the individual level so that heavy and less used parts of the representative entities can be swapped in memory handling operations.
2009-10-14
- Browsing
- e-Science
  - ...
2009-10-13
- Browsing
- e-Science
  - e-ISS
    - ...
      - http://www.mygrid.org.uk/dev/wiki/display/neiss/Taverna+Workflow+Execution+Portet+Application
  - Meeting with Mark Birkin
    - ...
  - Meeting with Junaid Arshad
    - Agenda
      - Publications
      - Progress on NeSS Work Package 4.1
      - Termination of Contract
      - AoB
    - Notes
      - Publications
      - AHM Poster
        
        http://www.allhands.org.uk/
        
        Accepted
        
        Junaid offered to get it produced, if I would go and present it, - he will be away on business in China.
        
        I should register by 2009-10-15.
        
        Practice paper for a journal about Junaids experience of developing e-Infrastucture for Social Science
        
        Andy to write a section which introduces a variety of e-Social Science projects (3 or 4) from around the world to demonstrate that work e-Social Science work is on-going and perhaps highlight some of its special needs.
        
        CIMEarth
        
        Medical/epidemiological/geographical
        
        Observational studies of children/vulnerable people/criminal/activist groups/activities
        
        Highlighting range of security concerns
        
        Data confidentiality/resource protections etc...
      - Progress on Work Package 4.1
        
        GLS Server
        
        No Progress with this.
        
        The GLS Server has not worked to join data for England since the GDAS was modified...
        
        Getting MoSeS Portlets running on Sakai
        
        Juniad created a test bed instantiation of Sakai, but there have been problems moving the portlets to work with Sakai.
        
        Paul has given up on moving the MoSeS system to Sakai for the time being as he focuses on GENESIS development and deployment to LifeRay.
        
        Andy to liaise with Paul about this...
      - Termination of Contract
        
        I had to let Junaid know that there is no plan to extend his contract at the end of this month.
        
        He will be getting a good reference and I hope to work again with Junaid in the near future.
        
        Junaid has holiday left which means he has 8 days work left.
        
        Junaid will still be around, but I don't want to have to rely on him to deliver any work which means that all his work needs to be made accessible to me or other staff at the University of Leeds or the University of Manchester working on join projects.
        
        Along with documentation, this should be the focus of the last 8 days work
      - AoB
        
        Security Work Package
        
        Junaid presented this work to the DSS Group and got some comments back.
        
        Pascal is carrying some of this work forward integrating the Shibboleth attribute handling with GridRelay to offer service protection.
  - GENESIS
    - Scaling up by implementing File Caching
      - ...
2009-10-09
- Browsing
- Miscellanea
  - Learner driving on the University of Leeds VLE and ELGG
    - ...
    - http://echo.leeds.ac.uk/ess/echo/presentation/1a18aed3-51b0-493b-94d8-e1a4f2277403
- e-Science
  - e-ISS
    - ...
  - GENESIS
    - ...
2009-10-08
- Browsing
- Miscellanea
  - MASS
    - Added Paula and Jorge to the NCeSS Sakai Portal MASS Worksite
- Teaching
  - GEOG1300 Tutorial 1 +
    - http://www.geog.leeds.ac.uk/people/a.turner/teaching/geog1300/2009/
- e-Science
  - e-ISS
    - Getting NeISS Spatial Population Engine up and running
      - ...
  - GENESIS
    - Working on a file based agent store
      - ...
    - Review of (Simulation Modelling) progress
      - Traffic Simulation Modelling
        
        Can now use Travelling Salesman and Open Street Map to route people from home (nearest bit of road to OA centroid) to work (nearest bit of road to OA centroid) where there is a route and failing finding a route the people travel by a shortest straightest path...
        
        There are perhaps still too many journeys not on actual roads that the picture from this still does not look like the existing transport infrastructure...
        
        I am exploring several ways forward...
        
        Using alternative routing map with Ordnance Survey data and routing algorithms implemented in GeoTools...
        
        If no route is found then try the shortest straightest path but check for other routes along the way and re-route if one is discovered...
        
        Try to route to prefered routes although it is hoped that these preferences will emerge...
        
        It may be that some unfound routes will be found if a bigger portion of OSM is used...
        
        Carrying on regardless focusing on something else...
        
        Scaling up to Leeds
        
        I am implementing a file based data store allowing the storage and retrieval of data for millions of objects.
        
        Once this works for the Demographic Modelling, the Traffic Model can be refactored to use it
        
        This should allow for scaling up from the small example of Otley in Wharfedale, hopefully a model of Leeds can be developed during the time remaining...
        
        Workflow
        
        Generalise to any region and develop a user interface that allows the region to be selected and a traffic simulation run...
        
        Adding constraints to travel on transport networks and looking at congestion
        
        The traffic model is being designed to handle flow constraints, but a model which constrains the amount of traffic along any part of a network is yet to be run...
      - Demographic Simulation Modelling
        
        A basic a-spatial model has been developed
        
        A spatial model is wanted that factors in migration...
        
        Assigning people to specific buildings within each OA for home and work
        
        This was work considered with Andrew Crooks, but there has been little progress...
2009-10-07
- Browsing
- Miscellanea
  - Organising a Geomorphology contract with Ordnance Survey
    - ...
- e-Science
  - e-ISS
    - Project management meeting
      - NCeSS Sakai Portal : NeISS Worksite : Wiki : Andy Turner Notes From NeISS Meeting 2009-10-07
        
        http://portal.ncess.ac.uk/access/wiki/site/e-iss/andy%20turner%20notes%20from%20neiss%20meeting%202009-10-07.html
        
        Only available to NCeSS Sakai Portal : NeISS Worksite participants.
    - Getting NeISS Spatial Population Engine up and running
      - ...
  - GENESIS
    - Working on a file based agent store
      - ...
2009-10-06
- Browsing
- Miscellanea
  - Mentor chat with Gordon Mitchell
    - We were both missing the CSAP seminar which looked interesting...
- Teaching
  - Booking rooms for GEOG1300 Tutorial Group meetings
    - http://www.leeds.ac.uk/timetable/assets/map/index.htm
    - http://www.leeds.ac.uk/timetable/roombookings.html
    - http://www.teachingspace.leeds.ac.uk/
    - Blenheim Terrace SR (G.07) House No. 11-14
      - http://www.teachingspace.leeds.ac.uk/room_details.asp?ID=1-01-086-2704-GR%20-G07
      - In Building 41 on the map:
        
        http://www.leeds.ac.uk/timetable/assets/map/index.htm
    - ...
- e-Science
  - e-ISS
    - Getting NeISS Spatial Population Engine up and running
      - ...
  - GENESIS
    - Working on a file based agent store
      - ...
2009-10-05
- Browsing
- Miscellanea
  - Completed Fire Safety training
    - http://www.leeds.ac.uk/safety/e-learning.htm
- Teaching
  - Booking rooms for GEOG1300 Tutorial Group meetings
    - ...
- e-Science
  - GENESIS
    - Working on an agent store
      - ...
2009-10-02
- Browsing
- Miscellanea
  - PC Crash :-(
  - Chat with Ant Beck about LUDOS and getting metadata from Geogrpahy harvested
    - I suggested Ant talk to Tim Baker, Lee Brown, Steve Carver, Andy Evans and get something in the newsletter again via David Appleyard.
    - Recently I noticed more openness with regard data from rainfor and trobit:
      - http://www.geog.leeds.ac.uk/projects/rainfor/pages/project_eng.html
      - http://www.geog.leeds.ac.uk/groups/trobit/
- e-Science
  - Catch-up meeting with Mark
    - ...
  - e-ISS
    - Manchester Taverna
      - http://rpc103.cs.man.ac.uk:8082/web/guest/taverna
    - Getting NeISS Spatial Population Engine up and running
      - Excited to see something called NeISS Spatial Population Engine Client appearing in Portlet Container Driver...
      - ...
  - MoSeS
    - UKDA/ESDS Submission
      - http://www.esds.ac.uk/aandp/create/ukdadeposit.asp
      - ...
2009-10-01
- Browsing
- GEOG1300
  - Thinking up some questions and collective exercises...
    - What is:
      - a map?
        
        a translation?
        
        an abstraction?
        
        relational?
      - data?
      - information?
      - understanding?
      - semantics?
      - knowledge?
    - What is a geographic/geographical map?
    - What is geographic/geographical:
      - data?
      - information?
      - understanding?
      - semantics?
      - knowledge?
- e-Science
  - e-ISS
    - Getting NeISS Spatial Population Engine up and running
      - Trying again with a new glassfish
        
        Downloading via http://java.sun.com/javaee/community/glassfish/
        
        Installed and started new Glassfish 2.1
        
        Installed open portlet container into new Glassfish
        
        Removed all servers from Netbeans
        
        Added the new Glassfish server
        
        Resolved project to use it
        
        Clean and build...
        
        Something still wrong...
      - Trying again with new Netbeans and having removed all .netbeans directories from user home directory
        
        Install Netbeans 6.7.1 and add all plugins
        
        Install Derby
        
        http://db.apache.org/derby/
        
        Configure Derby
        
        Services: Databases: Java DB: Properties:
        http://wiki.netbeans.org/NetbeansDerbyDir
        
        Install glassfish and liferay bundle
        
        http://www.liferay.com/web/guest/community/wiki/-/wiki/Main/Liferay+with+GlassFish
        
        Install Open Portlet Container in glassfish
        
        Open Netbeans project and configure
        
        Deploy...
2009-09-30