-
-
- Browsing
- PhD
- I must get round to revisiting this work else the project will die!
- Geomorphometry
- Identifying where OutOfMemoryError is being thrown:
- Does the abstract method in the abstract class require memory to pass things through it?
- If the memory handling wrapping simply needs to be in the abstract class, this is actually not a problem and should make the code more consise.
- Well maybe it does, but I think now that any method call requires some memory that is not reserved by the JVM.
- Maybe I have failed to reinitialise the memory reserve somwhere... Well, that may be the case, but that is not the problem in this instance.
- It is looking increasingly likely that any method call requires some memory that is not reserved by the JVM:
- A key thing to know is how much memory it takes to call a method and enter a try block.
- One could have a reserve of this size or larger and free it up prior to calling a method (a method that try{}catches(OutOfMemoryErrors) and then swaps chunks of grid data to disk) and then initialise it again as the first step in that method.
- This would be ugly and inefficient, but it should work and be robust.
- If this assessment is correct, then there is a way to internalise the memory handling, but that every method call will get a little ugly.
- Paul has kindly said that he will look at this for me and give his expert opinion :-)
- I could still do with a program that fails faster to diagnose this as it currently takes over 2 hours for the test to reach failure...
- OGC Grid Collision, MoSeS
- Meeting with Paul
- At AHM Paul met an NGS person based at Oxford who is prepared to offer us use of a 128 node machine and several TB of storage.
- Paul to put me in contact.
- Human Resources for MoSeS
- Summarised my meetings with Mark and Louise
- James Farnhill to push security contacts TO Junaid.
- Going through the process of setting up geo-s12 from a reboot
- It seems that working as tomcat has problems with the way things are set up.
- Paul to investigate if GridSphere deployed as root rather than tomcat makes a difference.
- Some part of Gridsphere was built or deployed as root and not tomcat user.
- It has now all been re-done as tomcat and everything works fine.
- MoSeS
- The draft paper document Mark provided me with on 2007-09-06 was as a result of Rob Proctor and Alex Voss asking us (MoSeS) to collaborate on an Asia/Europe project. It is not entirely clear what needs to be done, and who needs to do it, but it does seem likely to be: a) strategically valuable, and b) not very lucrative - small thousands at best. So in Marks estimation, if it came off, it would be most likely workshops, meetings etc...
- Beowulf processing check:
- Received and written result with 149 HSARDataRecords and 0 ISARDataRecords for 20UFGX0008 with fitness 10.02158792529489082459
- Sending input 105492 with ZoneCode 20UFGY0009 to 20
-
- Browsing
- SoG
- Geomorphometry
- Identifying where OutOfMemoryError is being thrown:
- try {
- _double0 = metrics1[ i ].setCell( cellRowIndex, cellColIndex, metrics1ForCell[ i ], handleOutOfMemoryError );
- } catch ( OutOfMemoryError _OutOfMemoryError ) {
- }
- Now,
- metrics1[i] is either a Grid2DSquareCellDouble or a Grid2DSquareCellInt which have their own setCell method which should automatically Override the setCell method in their shared abstract class Grid2DSquareCellAbstract.
- _double0 is already initialised, so my understaning is that overwriting it requires no additional memory.
- The setCell methods in both Classes are supposed to handle OutOfMemoryErrors.
- This mechanism appears fine, for example Grid2DSquareCellDouble.setCell(long,long,double,boolean):
- public double setCell(
- long cellRowIndex,
- long cellColIndex,
- double newValue,
- boolean handleOutOfMemoryError ) {
- try {
- return setCell(
- cellRowIndex,
- cellColIndex,
- newValue );
- } catch ( OutOfMemoryError oome0 ) {
- if ( handleOutOfMemoryError ) {
- clearMemoryReserve();
- if ( true ) {
- ChunkID chunkID = new ChunkID(
- this.nChunkCols,
- getChunkRowIndex( cellRowIndex ),
- getChunkColIndex( cellColIndex ) );
- swapToFileGrid2DSquareCellsGrid2DSquareCellChunkExcept( chunkID );
- }
- initMemoryReserve();
- return setCell(
- cellRowIndex,
- cellColIndex,
- newValue,
- handleOutOfMemoryError );
-
- } else {
- }
- }
- }
- I did notice a difference in that the Grid2DSquareCellInt.setCell(long,long,double,boolean) method was declared as:
- Whereas in Grid2DSquareCellDouble.setCell(long,long,double,boolean) "@Override" was not part of the declaration.
- The "@Override" is not necessary and I've recompiled with out it to see if the problem goes away.
- Well, it seems that removing the @Override has not helped. In a way that is good, as I would be worrying about Java if this was the problem!
- Trying something else:
- The setCell method returns something, in this case, a primitive. It could be that in organising this (prior to stepping through the method) pre-allocated memory is not being used and new memory allocation takes place. This would be bad for my Grids code as I rely on this not being the case. It may be that this only happens if the return type is primitive rather than an Object, that would be wierd, but not too hard to sort out.
- Anyway, setCell need not return anything, getCell can be used instead, maybe this is what the problem is...
- I refactored my code so that nothing is returned and set my test running again to see if I hit the same problem... If I don't, then there could be an equally big problem in that getCell in such a circumstance would fail too (and I would be worrying about Java if this was the problem).
- Well, the result is; I hit the same problem.
- Surely the abstract method in the abstract class does not require memory to pass things through it?
- Maybe I have failed to reinitialise the memory reserve somwhere...
- Surely there is a way to internalise the memory handling...
- I could do with a program that fails faster to diagnose this as it currently takes over 2 hours for the test to reach failure...
- i = 6
- cellRowIndex = 0
- cellColIndex = 1875
- chunkCellRowIndex = 0
- chunkCellColIndex = 0
- chunkRowIndex = 0
- chunkColIndex = 15
- chunkNrows = 125
- chunkNCols = 125
- OGC Grid Collision, MoSeS
- MoSeS
- Beowulf processing check:
- Received and written result with 64 HSARDataRecords and 0 ISARDataRecords for 19UHHX0005 with fitness 13.16557638139428638226
- Sending input 104200 with ZoneCode 19UHHY0014 to 2
-
- Browsing
- PhD
- Registration
- Phil Rees signs form and asks me to talk to my supervisors about planning a first Research Support Group (RSG) meeting
- Geomorphometry
- Identifying where OutOfMemoryError is being thrown:
- The DEM José gave me on 2007-09-04 hit an OutOfMemoryError.
- I think this is a bug that has failed things before.
- I recall having the problem with some data Steve and some data Alice gave me too...
- I want to get this fixed!
- I wondering if unreserved memory is attempted to be used in the execution of if and for blocks in evaluating if the condition is true...
- If this is the case, a major refactor will be required...
- Searched online for information of such problems, but not turned anything up. The general advice seems to be to use profiler software to track things down.
- Perhaps I should develop a test program to look into this...
- Recall a simple test program I wrote years ago to assess Runtime.getRuntime().totalMemory() and Runtime.getRuntime().freeMemory().
- I still don't understand the results from running it!
- Why should the "free memory" increase after having allocating resource to an array? Surely it really decreases...
- Maybe I just identified the problem.
- For some reason I had declared the _ChunkID field in Grid2DSquareCellChunkAbstract as transient so that when a Chunk was reloaded it either needed to be re-initialised or there could be a problem. I think the problem surfaced as a Grid2DSquareCellChunkAbstract
- Changing the transientivity had no effect in that the OutOfMemoryError problem was still there, so the fields transient status was reverted.
- Giving this some thought...
- DSOnline
- OGC Grid Collision, MoSeS
- MoSeS
- Beowulf processing check:
- Received and written result with 115 HSARDataRecords and 0 ISARDataRecords for 18UKGF0008 with fitness 5.77421345524542268052
- Sending input 102737 with ZoneCode 18UKGG0018 to 27
- Browsing
- DSOnline
- Liason with Swen regarding the About page changes he made for the Games and Simulation Topic Area.
- Reviewed a paper "NMCRC-DIOXINATOR - A Program for the Simulation of Dioxin Emission and Atmospheric Dispersal from Waste Incineration" for COMPUTERS, ENVIRONMENT AND URBAN SYSTEMS.
- This was the second time I reviewed this work and all the previous recommendations had been taken into account and the paper had been significantly improved.
- I recommended acceptance subject to two very minor edits and a careful check on Figure/Table numbers.
- OGC Grid Collision, MoSeS
- MoSeS
- Beowulf processing check:
- Received and written result with 116 HSARDataRecords and 0 ISARDataRecords for 18UCGM0022 with fitness 38.45153250727098550109
- Sending input 101438 with ZoneCode 18UCGN0017 to 12
- Browsing
- SoG
- Idea for Level 3 module research placement project
- River Basins Processes and Management Research Cluster / Centre for Spatial Analysis and Policy Research Cluster
- Geomorphometrics
- This project can be anything you'd like to do with this subject.
- In the next year we are going to start looking at so called "axis based metrics", such as profile concaveness/convexness.
- This project would suit someone who is interested in computer programming and GIS.
- OGC Grid Collision, NCeSS e-Infrastructure, MoSeS
- Draft Job description for work on the NCeSS MoSeS node to develop e-Infrastructure for the Social Sciences:
- Summary of Post
- This role is to assist in the development of e-Infrastructure for the Social Sciences working with the NCeSS MoSeS node.
-
As a technical advisor and solutions developer you will investigate options for the architecture and work to implement those deemed appropriate.
We are not developing a service, but the software architecture (part of the e-Infrastructure) required to support a service.
This must be secure, robust, extensible, scalable and well document.
You will be part of a team of around five people supporting the NCeSS MoSeS node which is developing a demographic model of the UK population (from 2001 onwards) and use this in various planning and research applications.
- MoSeS
- Beowulf processing check:
- Started 100000 to 110001.
- OGC Grid Collision, MoSeS
- NCeSS e-Infrastructure for the Social Sciences - MoSeS Human Resources Meeting
- People
- Louise Dyson, Mark Birkin, Andy Turner
- Mark and Louise have been meeting recently to develop Job Descriptions for work on MoSeS aimed at Chengchao Zu and Louise Mackay in mind.
- I felt MoSeS would also benefit from having someone from a more computer science background to work particularly in view of the technical work on the NCeSS e-Infrastructure for the Social Sciences Project. I suggested Junaid Arshad might be interested in this work, particularly that relating to security and grid services.
- Andy to draft another Job description.
- Mark to contact Jie Xu to make him aware of what is being aimed at Junaid.
- GeoLinking telecon
- Attempting to get the Demonstarion portal (GridSphere) back up and running
- org.gridlab.gridsphere.portlets.core.admin.setup.GridSphereSetupPortlet.1 Portlet unavailable!
- Sent tail of catalina.out to Gridsphere-users list as requested by Oliver Wehrens.
- Browsing
- OGC Grid Collision, MoSeS
- Attempting to get the Demonstarion portal (GridSphere) back up and running
- org.gridlab.gridsphere.portlets.core.admin.setup.GridSphereSetupPortlet.1 Portlet unavailable!
- Ran 'ant install' in /home/tomcat/gridsphere-2.2.15/ staopped and started tomcat, but probalem persists)
- Noticed that the Gridsphere-users list I joined yesterday is [NO LONGER USED] (sigh!)"
- Joined the Gridsphere-users list and fired off a question.
- Catch up meeting with Mark
- Feedback on powerpoint slides for Malcolm Atkinson prepared by Mark:
- Mark wanted input to technical details on page 2:
- I pointed out that much of this slide is related to the SEE-GEO Poster for the upcoming AHM.
- Provided Mark a copy of the draft SEE-GEO Poster for the upcoming AHM.
- Explained to Mark about the GeoLinking work
- High level summary and technical details about how our GLS client and Michael's GLS Server operate.
- Mark asks why GridSphere and not Sakai:
- This was a decision Paul and I made during the NeSC Portals and Portlets workshop where both Sakai and GridSphere were showcased.
- There were several reasons:
- Advice from Chuck.
- GridSphere seemed more mature and was already JSR-168 compliant.
- The JSR-168 compliance meant that we were confident that we could shift portlet container (to Sakai or other) in the future.
- Sakai seemed to work with IFrames whereas GridSphere was all XSLT.
- We are still interested in Sakai as others in NCeSS are developing/using it.
- I mentioned what I new of the security work and outlined why security is important and how we might employ different forms of security to enhance MoSeS.
- MoSeS modelling
- IPS GA Modelling results comparison
- I explained what I had done with regard constraints (both control and optimisation).
- We looked at the latest modelling results.
- I copied the latest MSOA results to the transfer area again.
- Dynamic model
- Practically working
- IPS output is currently feeding model, but Mark keen to use GA output instead now.
- Meeting with Louise Dyson on Thursday about Human Resource for MoSeS (including e-Infrastructure work).
- GROWL meeting lancaster on 18th
- Andy to write a few sentences about SEE-GEO collaboration and MoSeS security work for Friday so Mark can incorporate it in the e-Social Science full Paper expanded from the accepted abstract.
- Details of the GeoLinking work can be found via the following URL:
- http://www.geog.leeds.ac.uk/people/a.turner/organisations/OGC/GeoLinkingIE/
- In collaboration with SEE-GEO and as part of the OGC GeoLinking Interoperability Experiment, MoSeS have been developing a Use Case and a GeoLinking Service (GLS) client.
- The MoSeS Use Case has ben supported by MIMAS (SEE-GEO and NCeSS e-Infrastructure for the Social Sciences project collaborators) making some of the Census Area Statistics we are using in our modelling work available via a Geospatially-linked Data Access Service (GDAS).
- The MoSeS GLS Client is built using Restlets (REST framework for Java) and is implemented as a JSR-168 portlet within a GridSphere portal framework.
- Security
- Junaid Arshad a completing MSc student and prospective PhD student in Jie's group) has been looking into security for MoSeS
- Generated a report on security options.
- Worked on implementing Shibboleth to devolve responsibility for access control of portal users in collaboration with John Watt from NeSC.
- Browsing
- SoG
- Meeting with José Muñoz-Rojas Morenés
- José is a visiting researcher from the University of Castilla-La Mancha (Toledo, Spain) GYGA.
- His research is based on exploring relationships between hydrogeomorphological processes, land use changes and landscape patterns on a hydrographical basin in central spain (Bullaque river basin, 2000 square kilometers in the Ciudad Real mountains range).
- Worked on defining landscape units based on geomorphological patterns for planning purposes.
- Visiting researcher in Plymouth last year.
- People:
- Scale 1:100000
- Data:
- Very precise 20m resolution DEM for study area.
- Software:
- I'm going to run the DEM data through the getMetrics1 code...
- OGC Grid Collision, MoSeS
- Attempting to get the Demonstarion portal (GridSphere) back up and running
- geo-s12 rebooted
- tomcat started
- org.gridlab.gridsphere.portlets.core.admin.setup.GridSphereSetupPortlet.1 Portlet unavailable!
- I wonder how to kick GridSphere (do I need to run ant install somewhere and restart tomcat?)
- Joined the Gridsphere-users list fired off a question.
- Event announcement
- UK eScience AHM JISC demos September 11th 1-2pm SEE-GEO (Secure Access to Geospatial Services)
-
EDINA, MIMAS, NCeSS and NeSC will demonstrate an application which integrates at run time MIMAS census data with EDINA boundary geographic data.
The application demonstrates the utility of web services and the value of separating the provision of core datasets from the wide variety of possible attribute data.
The technology used is based on OGSA-DAI combined with open interoperability standards from the Open Geospatial Consortium (OGC).
- Browsing
- YAWS
- Erlang
- GeoLinking geographic information Google Alert
- Data
- SoG
- Advice for Richard Coppell on Spatial Statistics
- Richard's request:
- Would anyone know, given a set of residuals of model predictions for geospatial data (it’s actually a rainfall-field-model residuals against rain gauge test point data), how can I calculate a combined % error instead of Root Mean Square, is there a standard? (accuracy of rainfall measurements are often quoted in % and never, I think, in RMS, relative or otherwise) Are there any good refs on this kind of thing? I take it standard deviations don't come into it since geospatial stats are different?
- Just to clarify: What you call error is the difference between two different sets of measures? - In this case two estimates of rainfall for the same time period.
- Assuming this is the case, then, the differnce can be sensibly measured with spatial statistics in a number of ways.
- Before I go any further in outlining options: Are your two sets of estimates for the same geometries (i.e. sets of points), or do the geometries differ?
- I expect the most interesting outputs will be map like surfaces generated using generalisations within fixed radius spatial neighbourhoods applying a distance weighting function.
- I probably have some software that will generate these for you.
- When do you need the results?
- Advice for Tara Athan on Web-based Mapping
- Tara's request:
- I am looking at some of the options for web-based mapping, including opensource as well as proprietary. I am particularly interested in this as an interface to a geodatabase with a temporal component. I would be grateful for any general advice in this topic. I have a few specific questions at the moment:
- There are a number of books published on the subject. Is there any one particular book that you might recommend?
- One of the options I have been looking at, MapServer, has the drawback that it is not thread-safe unless a Python script is used or additional safeguards are built in. I am wondering how severe a limitation this is in practice.
- The options I have discovered so far include:
- GoogleMaps, YahooMaps, Mapquest, MSN Virtual Earth MapTech MapHub MapGuide from Autodesk MapServer ArcIMS ASPMap GeoMedia from Intergraph terraIMS (subscription) geoXtract from terraIMS MapXtreme from MapInfo
- OSGEO is one organisation you should look to for advice on Free and Open Source solutions.
- What are you trying to do?
- You can add GeoServer/GeoTools/Community Map Builder/uDig as potential components to an open source Java GIS solution. You may also want a back end database such as postgres/postgis.
![[Validate RDF]](http://www.geog.leeds.ac.uk/people/a.turner/images/rdf_w3c_button.gif "W3C RDF Validation")
![[Validate CSS]](http://www.geog.leeds.ac.uk/people/a.turner/images/vcss.gif "W3C CSS Validation")
![[An image of Andy Turner]](http://www.geog.leeds.ac.uk/people/a.turner/a.turner.png)