- Events
- Planning to attend/attended:
- Others of interest:
-
-
- Browsing
- Miscellanea
- PC Crash circa 11:46
- Lost all edits to the paper for the Oxford e-Research 08 Conference I was writing this morning!
- Should save more often, especially since the similar recent PC Crash on 2008-08-19
- e-Science
- Made up blurb for the NGS demo at the AHM emailed to Gillian:
- Using The NCeSS MoSeS and Geographic Linkage Service interfaces
- This is a demonstration of progress by the National Centre for e-Social Science (NCeSS) Modelling and Simulation for e-Social Science (MoSeS) Node.
- The demonstration is in two parts:
- The portal/user interface to MoSeS applications (http://geo-s12.leeds.ac.uk:8080/gridsphere).
- The Geographic Linkage Service (GLS) developed as part of the NCeSS e-Infrastructure for the Social Sciences Project (http://www.geog.leeds.ac.uk/people/a.turner/projects/eInfrastructure/GLS/).
- The MoSeS applications are based on a UK demographic model which estimates population characteristics at an individual and household level on a yearly time step for 30 years from 2001. The development of the demographic model is computationally demanding and NGS computational power has enabled this to happen.
- The GLS is being developed as a service with a server based on NGS which allow users to link geometry data with attribute data via a linkage attribute as is a commonin geographical mapping. This can be a computationally intensive task and there are various security and service registering hurdles that are being overcome. This work has been done in collaboration with the JISC Grid OGC collision programme (http://www.jisc.ac.uk/whatwedo/programmes/eresearch_grid_ogc_collision.aspx).
- MoSeS
- Progress with recalculating results that were not correctly constrained:
-
- Browsing
- Miscellanea
- MASS Meeting
- The effects of local information and trading opportunities in a network constrained economy
- e-Science
- SEE-GEO
- MoSeS
- Progress with recalculating results that were not correctly constrained:
-
- Browsing
- Miscellanea
- Masters dissertation help for Sadhvi Selvaraj
- Continued from 2008-08-21
- Documentation and referencing
- Want a way to give direct reference type URL's to both the source code and the javadoc for a method:
- Given that javadocs are HTML then
<a name="#anchor"></a>
type addresses in the source code comments can be linked to in a browser rendering:
- e.g. #_Output(AbstractGrid2DSquareCell,File,ImageExporter,String[],ESRIAsciiGridExporter,boolean) or #_Output(uk.ac.leeds.ccg.andyt.grids.core.AbstractGrid2DSquareCell,%20java.io.File,%20uk.ac.leeds.ccg.andyt.grids.exchange.ImageExporter,%20java.lang.String[],%20uk.ac.leeds.ccg.andyt.grids.exchange.ESRIAsciiGridExporter,%20boolean) respectfully:
- http://www.geog.leeds.ac.uk/people/a.turner/src/andyt/java/grids/dist/javadoc/uk/ac/leeds/ccg/andyt/grids/process/Grid2DSquareCellProcessor.html#_Output(AbstractGrid2DSquareCell,File,ImageExporter,String[],ESRIAsciiGridExporter,boolean)
- http://www.geog.leeds.ac.uk/people/a.turner/src/andyt/java/grids/dist/javadoc/uk/ac/leeds/ccg/andyt/grids/process/Grid2DSquareCellProcessor.html#_Output(uk.ac.leeds.ccg.andyt.grids.core.AbstractGrid2DSquareCell,%20java.io.File,%20uk.ac.leeds.ccg.andyt.grids.exchange.ImageExporter,%20java.lang.String[],%20uk.ac.leeds.ccg.andyt.grids.exchange.ESRIAsciiGridExporter,%20boolean)
- In the java class file source code my browsers do not have recognise these as links:
- Developing Neural Network code:
- Developing Roof Classification Example...
-
- Browsing
- Miscellanea
- Masters dissertation help for Sadhvi Selvaraj
- Continued from 2008-08-20
- Refined GenerateRoofData Java class to generate some synthetic roof data:
- Resizing and rescaling methods developed.
- Started to develop Neural Network code:
- Isolate NeuralNetwork from other code.
- Update to use latest grids...
-
- Browsing
- Miscellanea
- Masters dissertation help for Sadhvi Selvaraj
- Continued from 2008-08-19
- Refined GenerateRoofData Java class to generate some synthetic roof data:
- Resolved issues to do positioning of ridges.
- Resizing and rescaling the input:
- Aggregation and Disaggregation can be done in many ways!
- This is to be done as the data is created.
- Need to get Jin's neural net code released properly as open source so that we can make good use of it...
- e-Science
- The Oxford eResearch Conference 2008 developing my paper submission:
- MoSeS
- Upgrade to Jdk1.5.0_16, but still encounter the following running CheckAndFormat:
- Exception java.lang.OutOfMemoryError: requested 167772160 bytes for GrET* in C:/BUILD_AREA/jdk1.5.0_16/hotspot\src\share\vm\utilities\growableArray.cpp. Out of swap space?
- Try with jdk1.6.0_07:
-
# An unexpected error has been detected by Java Runtime Environment:
#
# java.lang.OutOfMemoryError: requested 167772160 bytes for GrET in C:\BUILD_AREA\jdk6_07\hotspot\src\share\vm\utilities\growableArray.cpp. Out of swap space?
#
# Internal Error (allocation.inline.hpp:42), pid=976, tid=2956
# Error: GrET in C:\BUILD_AREA\jdk6_07\hotspot\src\share\vm\utilities\growableArray.cpp
#
# Java VM: Java HotSpot(TM) Client VM (10.0-b23 mixed mode windows-x86)
# An error report file with more information is saved as:
# C:\Work\src\andyt\java\projects\MoSeS\hs_err_pid976.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
- A more useful error message which lead me to write a simple test case to reproduce the problem and filed a bug report with Sun...
-
- Miscellanea
- PC Crash 14:45
- Power interupt?
- With this I lost Blog edits for some of the last two days:
- I should save this file more often!
- Masters dissertation help for Sadhvi Selvaraj
- Continued from 2008-08-18
- Together we wrote a Java class to generate some synthetic data:
- javadoc
- src
- There are still some issues to be resolved before this is good.
- The synthetic data will require further processing to be used as input into a neural network model:
- Each roof will need to be converted to a standard dimension by aggregation or interpolation.
- All the values will need to be rescaled and it is probably best to have a minimum of 1 and maximum of say 10.
- This is probably best done with another class.
- Need to look at Jianhui Jin's (Jin) neural network code to see how we can use it:
- I will email Jin to let him know what we do...
- e-Science
- MoSeS
- Awaiting new Java install...
-
- Miscellanea
- Masters dissertation help for Sadhvi Selvaraj
- Continued from 2008-08-13
- Began wrting a Java class to generate some synthetic data.
- e-Science
- MoSeS
- Awaiting new Java install...
-
- Browsing
- e-Science
- NeSS
- MoSeS
- Progress with calculating missing results:
- Progress with recalculating results that were not correctly constrained:
- Code new Fitness measure that takes into account some transport variables like commuting distance/mode and car ownership.
-
- Browsing
- Miscellanea
- Masters dissertation help for Sadhvi Selvaraj
- Continued from 2008-08-11
- Brainstorm
- I think Neural Networks is the way to go and here are some steps to take:
- Select various geomorphometrics and other inputs that look like they might help distinguish the different types of roof.
- Divide each building footprint into oblong shapes.
- For each oblong rescale each variable from Step 1 into some fixed rasterisation (e.g. 64 by 32).
- Classify the scaled raster from Step 3 using a well trained Neural Network.
- We also considered the surface area/footprint area of the roof of the buiding and if this was a useful metric to distinguish different types of roof.
- On reflection, even finding the ends of the roof from the building footprint is non-trivial.
- I provided a copy of Jianhui Jin's MedAction Neural Network and Fuzzy Inference code.
- I obtained some sample data, but don't really know yet what to do with it... (I only have it to help Sadhvi)
- e-Science
- MoSeS
- Progress with calculating missing results:
- Progress with recalculating results that were not correctly constrained:
-
- Browsing
- e-Science
- SEE-GEO
- Created a new SourceForge Project
- Project Name:
- Unix Name:
- Public Description:
- Registration Description:
- The Open Geospatial Consortium (OGC) GLS Specification is still a moving target in 2008, but it is subject of the formation of a Standards Working Group. Version 0.11.0 integrates a separated specification on the GDAS data encoding. During the OGC Geolinking IE the specifications for GLS and GDAS were separate and progressed to Version 0.10.2. GLS is now used as an abbreviation for Geographic Linkage Service. As part of the JISC Grid OGC Collision collaboration client server software is been developed and security issues are being considered. The client is essentially a JSR-168 portlet that passes parameters to a server which connects to two data resources and provides the output (linked data) back to the client. There is communication and various security that can be detailed to enhance such a service. This project is about evolving thiss initial efforts and feeding the standards efforts of the OGC and various projects that might have use of such a GLS.
- Meeting
- Minutes emailed by Chirs Higgins the same day :-)
- NeSC Glasgow have GLS Cient up and running at the following:
- MoSeS
- Progress with calculating missing results:
- Progress with recalculating results that were not correctly constrained:
- Refactored code to integrate some CheckAndFormat and ReloadAndOptimise methods into IndividualCensus.
-
- Browsing
- Miscellanea
- CSAP Meeting with Calum Carson
- Nominating Oliver Duke-Williams to ESRC Research Resources Board:
- Masters dissertation help for Sadhvi Selvaraj
- Continued from 2008-08-08
- HHLL and slopyness metrics help distinguish hipped and gable end roofs.
- Given the resolution and accuracy of the DSM data, it is only clear for larger roofs.
- How to automate the assignment of roof type to building footprint?
- Reasoned that it is key to be able to overlay the DSM data and the MasterMap data:
- Suggested using Arc Gridwarp command having identified some points to warp
- Being able to recognise the end of a building and clip it off square is a help.
- For each building really we are after a count of the number of gable and hip ends for starters, although these could be assigned to edges.
- Suggested more extensive use of synthetic data to investigate resolution and error effects on different metrics.
- e-Science
- MoSeS
- Noticed that in IndividualCensus.init_IncludedVariablesHashSet() only malesAge80AndOver and femalesAge80AndOver are currently used.
- I think this was changed from a full set of age groups when results for Leeds and for the Genetic Algorithm and Iterative Proportional Sampling methods/results were being compared:
- That is before the missing results have been calculated.
- I'm unsure about what other results have been optimised in this way.
- It is perhaps safest to say that the first Individual and Household UK dataset put together will be a mixture of optimisations and significant provenance will be lost, so it may be very unlikely that the same result can be reproduced easily.
- Still, further re-optimisations based on this first dataset are desirable to produce several UK datasets.
- If in the future full provenance is deamed vital, then the results will have to be reproduced from scratch.
- Progress with calculating missing results:
- Progress with recalculating results that were not correctly constrained:
- Code new Fitness measure that takes into account some transport variables like commuting distance/mode and car ownership.
-
- Browsing
- Miscellanea
- Masters dissertation help for Sadhvi Selvaraj
- Sadhvi has some high resolution data for Portbury near Bristol in the UK:
- Can the difference between gable and hip end roofs be recognised and MasterMap building footprints classify accordingley?
- http://www.cmhpf.org/kids/Guideboox/RoofTypes.html
- Suggested generating and looking at output of the geomorphometrics program I lectured her about on GEOG5060 earlier in the year.
- Looked at some output together and thought about the next steps.
- Looking forward to seeing where this goes over the coming weeks :-)
- Nominating Oliver Duke Williams to ESRC Research Resources Board:
- e-Science
- MoSeS
- Progress with calculating missing results:
- Recalculation code/processing for if constraints somehow not met:
- Two potential problems:
- Wrong number of households:
- _ZoneCode 00AAFA0002 _HPHRPSize 61 != _AllHouseholdsTotal 62
- Wrong total communal establishment population:
- _ZoneCode 00AAFE0001 _CEPSize 0 != _CommunalEstablishmentPopulation 3
- The problem here was that in constraining originally there was error and male counts were taken as female counts.
- Progress with recalculating results that were not correctly constrained:
- Code new Fitness measure that takes into account some transport variables like commuting distance/mode and car ownership.
-
- Browsing
- e-Science
- MoSeS
- Progress with calculating missing results:
-
- Browsing
- e-Science
- MoSeS
- Something is not right with the ReloadAndOptimise routine:
- When re-evaluating output, it is not being measured with the same fitness...
- It seems that somewhere the population is increasing in size. Is it that all household population are being treated as HRPs somehow in reloading?
- Something along these lines was occuring and the problem resolved.
- Recalculation code/processing for if constraints somehow not met:
- For some Output Areas (e.g. 00AAFA0007) the Communal Establishment Population (CEP) is not passing control constrained test for some reason:
- Upon recalculation CEP still not constraining.
- Writing out constraints:
- Identified and fixed error in constraint calculation test.
- The constraint was calculated by doubling female population rather than adding female and male population.
- This was perhaps a cut and paste error, but it reminded me that something like this happened before, but would have effected the constraints as used in generating the population results in the first place.
- I think this problem was identified mid way through the calculation of the UK data and I resolved to sort it out at this stage.
- Indeed some Output Areas (e.g. 00AAFE0001) now not constraining... so more work for the coming days...
- Code new Fitness measure that takes into account some transport variables like commuting distance/mode and car ownership.
- A preparatory look at what data might be used:
- Progress with calculating missing results:
-
- Browsing
- Miscellanea
- Probation meeting with Junaid Arshad
- Completed the probation documentation as a paper and pen exercise.
- I have recommended Junaid complete probation early as of 2008-09-01.
- e-Science
- MoSeS
- Progress with calculating missing results:
- NeSS
- Meeting with Junaid:
- Tested geo-s12 GLS Client works with NGS VM at gls.vidar.ngs.manchester.ac.uk
- NCeSS VM should also work, but we need Alex Voss to do some configuration.
- Junaid spent a lot of time debugging and working through the GLS operation
- This is good because Junaid now fully understands how it works.
- It is slightly bad as some configuration had not taken place that Junaid had thought when debugging.
- Junaid to liaise further with Shiv Kaushal and Alex Voss about DN checking and blocking users without UK eScience Certificates.
- Juniad to ensure the NGS GLS Server will only communicate with the various GLS Clients using IP address matching:
- This should provide sufficient interim security.
- We discussed further aspects of developing the GLS as an NGS service.
- I forwarded Junaid the email I sent to Mike Jones and Chris Higgins on 2008-07-16.
- Junaid thinking that in terms of resource monitoring then Matt Ward is who we will be dealing with :-)
-
- Browsing
- e-Science
- MoSeS
- Progress with calculating missing results:
-
- Browsing
- Miscellanea
- e-Science
- MoSeS
- Progress with calculating missing results:
- The following is a closely related publication to the demographic work:
- References
![[Validate RDF]](http://www.geog.leeds.ac.uk/people/a.turner/images/rdf_w3c_button.gif "W3C RDF Validation")
![[Validate CSS]](http://www.geog.leeds.ac.uk/people/a.turner/images/vcss.gif "W3C CSS Validation")
![[An image of Andy Turner]](http://www.geog.leeds.ac.uk/people/a.turner/a.turner.png)