Andy Turner's xhtml2.0 Blog 2006-11 Page @ School of Geography, University of Leeds

Andy Turner's Blog
2006-12-01
2006-11-30
- SoG Seminar
  - Sue White from Cranfield
  - Moorland management, climate change etc...
  - Good, thanks Sue :)
  - I wondered how much a factor the lack of a sustained winter coverage of snow and ice, or very harsh winter conditiond, is why Yorkshire Winter moor has become more heather infested... I pondered this with Bri and Katherine...
- Helped Charlotte Rawcliffe's group in Skin Microbiology by volunteering in a skin washing experiment
  - They are testing an experiment design involving washing the skin multiple times and examining the microbiology.
  - It is very difficult to place a ring in exacly the same place three times in a row. Using concentric rings with a scaled and known area and volume decrease seems more sensible to me...
- MASS meeting
  - Got permission from Mike Crabtree to use FreeCiv in GIS/Masters lab.
  - Session on FreeVic that I co-ordinated seemed to go well.
2006-11-29
- MoSeS meeting with Martin and Mark
  - Like is still not being compared with like in the regressions of observed v expected
    - A couple of logic errors in the aggregation of results were spotted.
    - New results were posted at the following:
      - AgeGroupHouseholdCompositionEmploymentHealthConstraints ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_10000_10_100_12_10_MarkOutput2OA_RegressionReport
      - AgeGroupHouseholdCompositionEmploymentHealthConstraints ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_10000_10_100_12_10_MarkOutput2MSOA_RegressionReport
- MASS meeting preperation
  - FreeCiv
    - Email thread on comparing the real and the game
- ESRC Understanding population trends and processes � a secondary data analysis initiative funding call
  - Should CSAP and MoSeS be targetting this?
  - What's going on?
- OGC Newsletter - November 2006
- Steve Taylor enquiry about Grids.
  - For Nidderdale AONB project with Steve Carver
- Provided copies of Spatiality in Design book to UCC... Thanks Bobbie... Feedback forthcoming.... :)
2006-11-28
- NIEeS Grid GIS Working Group Meeting
- Registered for OGCNetwork
- Registered for OGC December 2006 OGC Technical Committee Meeting - San Diego, California
2006-11-27
- DSonline Gaming and Simulation
  - Telephone call with Swen:
    - Hopefully Brooke gets back to us before Wednesday so we get content ready on Wednesday/Thursday and linked in by the deadline (2006-12-01).
    - Swen to email broader editorial group to ask for feedback on the content in the word document drafts of the web pages.
- PhD
- MoSeS
  - Martin wants me to summarise what was good about the results that made my week last week.
    - I'll get on to this after I've obtained the missing result... got the result... integrating it... published:
      - AgeGroupHouseholdCompositionEmploymentHealthConstraints ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_10000_10_100_12_10_MarkOutput2OA_RegressionReport
      - AgeGroupHouseholdCompositionEmploymentHealthConstraints ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_10000_10_100_12_10_MarkOutput2MSOA_RegressionReport
  - CCSR Seminars Page - Autumn 2003 interesting for the talks about SCAM and disclosure control.
- Can laboratory studies really explain processes that operate in the field? Jo Clark
  - Accepted a meeting on the 2006-11-30 12:00-13:00 to discuss...
  - IMHO both laboratory and field based studies can be enhanced with computer based visualisation and modelling work. No matter what is done, I think that one of the key things is online dissemination of the data collected in an OGC standards compliant manner. I think that is very important so that all can build on the work and it would include a full metadata detailing how the data was abstracted and processed. Additionally I would hope that the research would embrace open source software.
2006-11-24
- CASA Working Papers
  - Principles and Concepts of Agent-Based Modelling for Developing Geospatial Simulations
  - Exploring cities using agent-based models and GIS
  - Imagining the Recursive City: Explorations in Urban Simulacra
  - Simulating Emergent Urban Form: Desakota in China
  - Spatial variations in road collision propensities in London
  - And theres many more great looking papers too :-)
- ONS Super Output Area Geography
- Task Force Action Points 2006-11-06 ... Hmmm interestingly, my Firefox browser has a problem with this URL ...
- DSOnline Gaming and Simulation
  - Stub created for home page.
- Gordon Conference next year on Coupled Biogeochemical Cycles
- Need to distill some of the information in this page and from the last few months of blogging:
  - Start with MoSeS stuff...
- CEH Data Holdings
- NERC Climate and Land-Surface Systems Interaction Centre - Darn account creation seems to be deactivated... Sigh!
- Registered for NERC Soil Biodiversity Thematic Programme Data Discovery and Download Facility
- MoSeS
  - Meeting with Junaid Arshad
    - I gave the low down on MoSeS and geography from my perspective.
    - 4-5 page extended essay focussing on Grid Computing and Virtual Organsiation aspects of MoSeS.
      - Focus on Reductionism and All Systems Thinking.
      - It would be good if Junaid were to talk with all on the MoSeS team.
      - I am very pleased this collaboration seems to be taking place and am happy to spend some time helping out :-)
    - Actions
      - Junaid to get a personal web page and link to his own as a means for us to see what each other are doing and communicate and develop an expostition of this work.
      - Andy:
        
        To develop MoSeS collaboration web page for this work.
        
        With our first set of actions complete Andy will make sure the Principal Investigator on MoSeS (Mark Birkin) is aware of our collaboration.
  - Population Initialisation
    - Urgh... Latest result that I pulled early looks rubbish... Maybe that is just because I pulled it early... Hmmm, no, that is clutching at straws... Something has probably gone wrong... I wonder what will happen when I tell Mark and Martin...
    - Result! It has worked in the main... Phew! One of the processes hung trying to send 00DAFC0066 result back so that is why the OA result looked rubbish... The MSOA result looks good... As a temporary fix, I stuck a null record in for the missing result... The results are online:
      - AgeGroupHouseholdCompositionEmploymentHealthConstraints ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_10000_10_100_12_10_MarkOutput2OA_RegressionReport
      - AgeGroupHouseholdCompositionEmploymentHealthConstraints ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_10000_10_100_12_10_MarkOutput2MSOA_RegressionReport
    - Set the MSOA constraint and optimisation result going... I estiomate this will take 2 weeks on the available half of the SoG Beowulf
2006-11-23
- I am informed that eLGG is University of Leeds prefered blogging software... Well it is free and open source :) ... Hmmm I could install it, but I won't ... Will it ever be something I use ... Hmmm ... I wonder ... I think I'll stick to blogging this way for the time being ... I had been considering a switch to using WordPress now I have a better idea about what is involved in hand knitting per se...
- Soils - From Catchment to Crumb
  - Sheila Palmer
  - Key Literature
  - MLURI
  - Rothamsted
  - A list of some soil data resources
  - Need for better modelling, spatio-temporal change work, upscaling - downscaling and uncertainty.
  - Dissemination of data is poor - even aggregated generalised data, let alone the raw data from sensors and observations.
  - Knowledge about how to do soil surveys is at a risk of being lost!
  - Miscellaneous links:
    - NERC Climate and Land-Surface Systems Interaction Centre - Darn account creation seems to be deactivated... Sigh!
- Concerned about being able to validate my XHTML� 2.0 Web Pages I did a search:
  - Still doesn't seem there is anything from W3C...
  - XHTML� 2.0 W3C Working Draft 26 July 2006
  - Email thread from July 2005 where someone was asking about an XHTML2.0 validator.
- PhD
2006-11-22
- PhD
- WUN Spatio-temporal GIS Seminar
  - Tried for ages to connect without success :-(
  - Hopefully Tim will help me find out why.
- Planet Earth Under Threat Blog
  - I made a post:
    - Thank you. I listened to the programme, but didn't hear every word. I was hoping it would make more of the growing hot deserts (I'm sure they are) when there was talk about habitat change. Maybe this will be covered in a subsequent broadcast. I will be listening :-)
      
      I don't think the argument about losing tundra to forest and little white fox to big red fox in Europe is going to help change the majority public into living a more sustainable lifestyle... But then it might and I don't have all the answers as to what a more sustainable lifestyle is... I was concerned about the bleeching of coral, but wondered if an adaption in the species is just araound the corner...
      
      I watched the latest in the Planet Earth T.V. series after I listened. It was great!
      
      I appreciate the concerns I read on the blog about travelling. My opinion is that BBC travel is justified when it is for quality news and documentary programmes.
  - Then emailed Lindsay to let her know...
- MoSeS
  - Martin is interested in CAS001vCAS002AgeGroupComparison
    - I should double check my calculations.
      - These seem to be fine!
    - Develop a means to report and map the outliers.
    - Analyse the reports and maps to identify any systematic errors.
2006-11-21
- Uploaded some files into the Stan Openshaw Collection
- MoSeS
  - Transport Application Meeting with Haibo and Belinda.
  - MoSeS Progress Meeting
  - 2006-11-20
    - DSonline Gaming and Simulation telephone meeting with Swen :)
      - We are to write an editorial together for the Magazine to launch the topic area.
        
        This can be based on the word document Swen circulated.
        
        Swen to rework and produce draft...
      - Swen also to:
        
        Email Brooke and find out about getting content online / status of our CMS accounts.
        
        Develop a list of links Games resources and organisations.
      - Andy to:
        
        Email Brooke and ask what the Magazine is
        
        Develop a list of links to Simulation resources and organisations.
        
        Journal of Social Simulation.
        
        Application domain specific...
    - MoSeS
      - Made a booking to use SoG Beowulf nodes 101-113 for 2006-11-24 to 2006-12-08.
      - Progress Report
      - ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_10000_1_100_12_10_MarkOutput2OA
      - ToyModel_SWR_OA_HSARHP_ISARCEP_0_10_10000_1_100_12_10_MarkOutput2MSOA
  - 2006-11-17
    - Retrieved Stan files from Maureen's disks that were given to me by John Dodds
      - Maybe some extra stuff there...
      - Plan to put this online and integrate with other papers etc.
    - Chat with John about his career.
    - Chat with Patrick about his career.
    - Diomidis D. Spinellis home page
  - 2006-11-16
    - Found Stan Openshaw's SoG web Home Page which had moved - Eeek! I've requested the webmaster makes a simlink or provides a redirect to fix all the URLs.
    - Rainford P, Masser I, 1987, "Population forecasting and urban planning practice: a case study" Environment and Planning A 19(11) 1463�1475 - Consider referencing in MoSeS work...
    - Short Research Profile Paragraph for the Staff Handbook provided by email to Louise Dyson
      - Andy is generally interested in methods for describing, modelling, simulating and analysing processes that interact in the ever expanding humanosphere - the space-time region that humans influence. Socio-economic systems can be adapted and influenced in various ways to mitigate environmental concerns and provide for sustainable development. Andy's research focusses on the characteristics, development and application of tools for assimilating, contextualising, modelling, simulating, analysing, visualising and disseminating geographic information. In particular, he is interested in developing tools for generalising and exploring data at higher levels of spatial, temporal and attribute detail for larger space-time regions. His research is collaborative and the outputs are openly available. The subject in which Andy specialises is known as GeoComputation.
    - MASS meeting
      - Looking at M5 method.
    - GAM
      - Zhi Jie, Zhang of the Deptartment of Epidemiology, School of Public Health, Fudan University, Shanghai, China is now using cluster.
        
        I've had various correspondence with over the last week.
        
        I'm planning to find out more about Zhi Jie, Zhang's work and how we can collaborate.
    - Research Assistant in spatial regression modelling and e-social science (ref.�12774)
      - I think this shows that the proposal I put online and was asked to remove was funded, so hopefully Claire won't be cross with me any more...
    - NeSC Events - "Agent based Grid Computing" scheduled for 2007-02-19 to 2007-02-20 could be good.
      - I posted a headsup to MASS.
    - MoSeS
      - OAs with large errors
        
        00DAFB0058 7991
        
        00DAFM0005 591
        
        00DAFN0008 5904
        
        00DAFN0018 2720
        
        00DAFN0010 9999
        
        00DAFN0011 1191
        
        00DAFN0013 13397
        
        00DAFN0018 2720
        
        00DAFN0021 11224
        
        00DAFN0023 475
        
        00DAFN0024 4018
        
        00DAFN0025 5825
        
        00DAFN0026 1789
        
        00DAFN0032 1794
        
        00DAFN0034 16897
        
        00DAFN0036 2570
        
        00DAFN0059 8450
        
        00DAFN0060 595
        
        00DAFN0062 851
        
        00DAFN0063 1226
        
        00DAFN0073 3437
        
        00DAFN0076 540
        
        00DAFR0060 1107
        
        00DAFR0067 428
        
        00DAGH0049 8794
        
        00DAGH0065 9749
  - 2006-11-15
    - CSAP Seminar
      - Challenges for Creating Time Series of Socio-Economic Data
        
        Paul Norman
        
        Thanks Paul :)
        
        Paul has done some consultancy with the electoral commission.
        
        Census TEst in May 2007
        
        Test form online...
        
        Plans to ask about multiple residencies/property ownership.
        
        DwP have individual microdata.
        
        In 1991 Census data was more out of student termtime than 2001.
        
        The General Solution IMHO is to allow Individual level data to be used with a control on publication that deals with the confidentiality/disclosure issue. A key to this is the Pseudo-anonymisation of records so that they can be linked.
        
        In 1991 Census data was more out of student termtime than 2001.
        
        Sent Paul http://www.geog.leeds.ac.uk/people/a.turner/projects/medalus3/ and http://www.geog.leeds.ac.uk/people/a.turner/publications/archive/OpenshawTurner2001.html for reference.
    - MoSeS
      - Grid Computing Will Let Town Planners Play 'SimCity' For Real
      - SimCity for Real Press Release at NeSC.
      - Chris from the Yorkshire media rang to ask some questions about MoSeS and SC2006.
      - ToyModel_SWR_OA_HSARHP_ISARCEP_0_100_10000_2_100_12_10_MarkOutput2MSOA_RegressionReport
      - ToyModel_SWR_OA_HSARHP_ISARCEP_0_100_10000_2_100_12_10_MarkOutput2OA_RegressionReport
    - US NOAA National Weather Service Data Available in GIS formats
  - 2006-11-14
    - Peter Balmforth is wanting us to help in the evaluation of the EVIE prototype.
    - PC crashed for the first time in ages. The culprit was Windows Media Player which I was trying to use to play a movie of some data visualisation... sigh...
    - OpenDX
    - VTK (Visualization ToolKit)
    - VISIT
    - NVIZ (part of GRASS)
    - MoSeS
      - Need to produce some stats for Paul...
        
        SoG Beowulf description
        
        The cluster comprises two master nodes controlling 13 and 15 nodes respectively.
        
        Each node has Dual 2.66 GHz Xeon processors 1.280MB DDR memory 40GB internal IDE disks.
        
        The processors are Dell Precision Workstation 450 CPUs.
        
        A Dell Poweredge 600 SC acts as the server for the cluster.
        
        Details of Beowulf usage for population initialisation
        
        For the UK at Output Area Level we have approximately 200000 tasks which can be run independently, each uses a little over 1GB of memory and takes about 5 minutes on a cluster node utilising one of the Dual 2.66 GHz Xeon processors.
        
        For the UK the entire run takes around 3 weeks utilising most of the SoG Beowulf.
      - I made a rather fundamental mistake...
        
        I was not loading the CAS002 census data correctly for England and Wales, seriously effecting the population initialisation.
        
        Originally I had got a joined CAS001 and CAS002 table from CASWEB, but in refactoring at some stage I messed up the CAS002 data loading by failing to get a separate CAS002 table file from CASWEB. Basically this mean't I was reloading CAS001 data thinking it was CAS002 data... It was hard to track down as the totals were fine, but the breakdown by gender and marital status was completely wrong...
        
        A comparison of tables CAS001 and CAS002
        
        MoSeS CAS001vCAS002AgeGroupComparison Page
        
        This now looks much more reasonable than it did a day ago!
        
        Interestingly it appears that an effect of SCAM is to cause a tick shape in these plots.
        
        There are other interesting differences...
  - 2006-11-13
    - MoSeS
      - ToyModel_SWR_OA_HSARHP_ISARCEP_0_100_10000_2_100_12_10_MarkOutput2OA_RegressionReport
      - ToyModel_SWR_OA_HSARHP_ISARCEP_0_100_10000_2_100_12_10_MarkOutput2MSOA_RegressionReport
      - In analysing results for Population Initialisation an inconsistency between two measures of the population age 80 and over has been identified:
    - http://openforecast.sourceforge.net/
    - https://www.seegrid.csiro.au/twiki/bin/view/AUKEGGS/FinalWorkshop
  - 2006-11-12
    - Bit of work on a Sunday!
    - Information Environment Service Registry
    - IESR OAI-PMH
    - The Open Archives Initiative Protocol for Metadata Harvesting
  - 2006-11-10
    - IMA 2007 Conference
      - Event of interest...
    - Steve Carver my new office neighbour and long standing friend, colleague and mentor popped in to say hello :)
      - We had a chat about Nuclear Waste and Wilderness research.
      - In the 20 year NIREX rule, documents are coming to light which should be cross referenced with Steve's PhD thesis.
      - I provided Steve with another box of Nuclear material, nothing radioactive or explosive though I don't think, but none-the-less could be useful.
    - MoSeS
      - Results for just optimising just on Age groups except the 80AndOver group.
        
        I patched together a set of results, but after a first look, it seems there was a problem as contol constraint observed and expected are not the same for all OAs.
        
        Re-running the analysis...
      - Another attempt to build GeoTools...
  - 2006-11-09
    - Offered to help Nick Malleson with OSGeo and asked him to sign up :)
    - MoSeS
      - Parallel processing
        
        There seems to be a problem with my use of waitany(Request[]) when collating the final set of jobs...
        
        For some reason the program hangs even after all processing is done on the slaves and the results sent.
        
        The strange thing is that in some cases some of the results are recieved on the master.
        
        To try and work around this I used the waitall(Request[]) method to collate the final set of jobs.
        
        Unfortunately this didn't work either...
        
        I wonder what I'm doing wrong...
        
        Next I tried to use waitany(Request[]) again but explicitely set to null a request after it has been recieved...
        
        This resulted in a NullPointerException...
        
        Not sure quite what tinkering solved this problem, but my program now seems to be working as expected :)
      - Data transfer...
      - Paul has maps working on the Portal :)
        
        Unfortunately Paul ended up devising a bespoke colour shader rather than using anything in GeoTools...
        
        Some of the useful GeoTools code of the past is obfuscated in the refactoring...
        
        Nevermind, we are getting to grips with things again...
      - For the time being Paul is not using the data handler I provided
        
        It would take a significant amount of code reorganisation.
        
        Other things are more important for SC2006.
      - Paul and I chatted about aggregating HSAR Data records
        
        I reckoned this code should live in an io package.
        
        Basically we want a HashMap or similar to count for each value of a variable for both individuals and households.
        
        Maybe I will get on to this while they are away at SC2006.
  - 2006-11-08
    - Virtual WUN Seminar on Spatio-temporal Modelling in GIS Environments: Agent models of population
      - Dick Ettema
      - Very interesting and relevant!
        
        I managed to plug both my Phd and MoseS and ask for pointers... I wonder if anyone will email as a result...
      - The talk was mainly about PUMA
        
        Model operates on a 500 meter grid although work is being done with 100 meter grids.
        
        PUMA: MULTI-AGENT MODELLING OF URBAN SYSTEMS. Dick Ettema, Kor de Jong, Harry Timmermans, Aldrik Bakema. Paper presented to the 45th Congress of the European Regional Science Association "Land Use and Water Management in a Sustainable Network Society" Vrije Universiteit Amsterdam 23-27 August 2005
      - Surfing links
        
        http://www.psych.lse.ac.uk/complexity/ICoSS/Papers/ConnectivityReportUB.pdf
        
        OPUS (Open Platform for Urban Simulation)
        
        Based on Urban Sim
        
        Key People
        
        Paul Waddel
        
        Eric Miller
        
        Kia Nagel
        
        Geosimulation
    - Registered, then gave up my place at this EcoHydrology Conference to Nanlin. I met Andy Jarvis, the one from Lancaster not the one from CIAT...
    - Emailed commons to respond to Jeff Drew's question about the differences in calculations of mean and standard deviation between Commons Math and Excel.
      - Not sure of the details of either Commons Math or Excel. But could this be a precission issue. Are something like BigInteger and BigDecimal used in the calculations? If not more precision will probably be lost for large calculations.
    - MoSeS
      - Minor modification to speed up data handler for ToyModel data.
      - Trying a run to optimise just on age and gender groups not including over 80s...
  - 2006-11-07
    - Email exchanges with Tim Motts about Cluster hunting and Crime Pattern analysis drew to a conclusion. Thanks Tim :)
    - MoSeS
      - Asked Mike about opening Port 80 on portal server so we can run Subversion and Maven. We will probably be needing a whole lot more for SRB.
      - Wrote a data handler for ToyModel data and made a new release of code.
  - 2006-11-06
    - PESERA project webserver at JRC has become defunct
      - Brian Irvine asked how to take the files and host them as web content.
      - I suggested using the SoG webserver and the following URL http://www.geog.leeds.ac.uk/projects/pesera
      - To do this Brian needs to contact David Appleyard and get the directory in place so that he has permission to write to it.
      - The files can then be copied there so that all can access them.
      - Next Brian needs to think about metadata.
      - Finally Brian can think about developing the original content, fixing links and making it standards compliant etc...
    - MoSeS
      - Emailed Paul to encourage him to extend some of my code for handling the data Mark gave him to help develop the portal :)
      - Still struggling to install GeoTools... It would be easy if I had administrator access ;-)
  - 2006-11-03
    - Computational Science & Discovery a new journal to add to my reading list.
    - MoSeS
      - Reverting to trying to use GeoTools Version 2.3.0 M0 as stable release 2.2.1 has a maven build failure problem...
      - Meeting with Paul and Martin to discuss graph results and population initialisation model
        
        Things todo:
        
        For age 80 and over and unemployment observed and expected are not in same order of magnitude. Why is this?
        
        Try weighting highly individual optimisation constraint parameters. Are these then right?
        
        Mapping at OA and MSOA
      - PhD 2006-11-03
  - 2006-11-02
    - PhD 2006-11-02
    - Emailed geotools user list with maven build problem.
    - Registered for a DSOnline web account.
    - Teleconference with Paul Townend, Chris Higgins and James Reid who were all together at EDINA :)
      - Discussing collaboration between UKBorders, NCeSS e-Infrastructure, OGC Grid Collision (SEE-GEO) and MoSeS
        
        James - UKBorders will have a working GML2 Geoserver based WFS next month (currently the WFS is non-OGC compliant)
        
        MoSeS can use the test/development server for consuming this service. Great! :)
      - A security review needs to be done for the Ordnance Survey Master Map (OSMM) service as part of SEE-GEO
      - The time frame is about the last quarter of 2007 with regard consuming a similar WFS service for displaying OSMM from EDINA.
      - UKBorders are thinking of developing a cartogram service. This would be great! :)
      - Paul gave a demo of MoSeS portlet.
      - Chris is to send a draft detailed workplan for the OGC Grid Collision project to me in the next few weeks.
        
        I am to go through this with Mark and make a committment as to when different bits of work will get done.
    - MASS Meeting
      - An introduction to using the Beowulf machine in geography.
    - Meeting with ITS to discuss ESRC Bid
      - Good meeting, seems they have thought of most things for setting up a national centre on a transport research theme.
      - The main thing the need form us geographers is a capabilities statement at this stage.
        
        Mark Birkin is going to organise this.
    - MoSeS
      - Starting to use GeoTools to generate some maps...
  - 2006-11-01
    - Occam's razor
    - Web Content Updates
      - My Home Page
      - This is Andy Turner's Organisations XHTML 2.0 Page @ School of Geography, University of Leeds.
    - MoSeS
      - Completed EPSRC and JISC funding project - Study of User Priorities for e-Infrastructure for Research (SUPER) questionairre
      - Completed first attempt to programatically process result plot generation for analysis of population initialisation using JFreeChart and Jakarta commons Maths
        
        Some example output: ToyModel_SWR_OA_HSARHP_ISARCEP_0_5_1000_3_30_12_20_MarkOutput2MSOARegressionReport
      - Provided some documentation for Paul on OGC Grid Collision and NCeSS e-Infrastructure projects in preperation for tomorrows meeting with Chris and James at EDINA.
  - 2006-10-31