GDQS - time off from the project

Geospatial Data Quality System

Since I had relatives visiting this week, I decided to take the week off from this project.  I knew I would be getting behind on work at my day job, so I spent a little time getting caught up on that.

I did have a little free time to do some digitizing in QGIS.  I'm going to need some data to QC once the project gets to the point of proving the capabilities.  It also gave me some time to explore the Web Map Services (WMS) that are available.  A search for Imagery WMS, gave me many options, but I chose the USGS EROS high res imagery (1 foot orthos).  I create a new QGIS WMS layer, picked a scale to collect from, and started digitizing:

This will give me a good dataset to QC later one.  Man, am I out of practice with digitizing!

Store achitecture

Tomorrow, I will continue building the Store architecture.

Geospatial Data Quality System - Project Info Table, GeoServer Catalog reloading, and Stores

Project Info Table - easily solved

When I left off last week, I was struggling with trying to display the Project metadata in a table on the Project Info Page.  Well, that was a case of getting frustrated and not thinking clearly and deciding I would tackle it during the next week.  I had a little free time to get on the computer Monday night, so I thought I would give it a shot, and was able to get the table display working within a few minutes.  Solving the problem was a matter of taking a break.

It's not the best implementation at the moment, since I need to add a new table row to the html template for each new attribute I add to the Project object (and it must match the attribute key):

"ProjectStatus" attribute added to the Project object

It is relatively simple on the Java side, and requires no code changes to display the new attribute value.  It simply iterates through the MetadataMap and creates the wicket Label objects to display.

I would like to simplify this further but this works for now.  In the future, I would like to have the whole table generated in code, not having to add anything to the html page.

GeoServer Catalog Reloading

As I was working on the MetadataMap table display, I created a couple of new Projects and noticed that those did not immediately display on the PorjectList page.  It would require restarting Tomcat to see any new projects on the ProjectList page, while I could see the new Project on the Workspace list page.  I figured there must be a way to refresh the cache of loaded project, so I dug through the code.  This is what I found:

That may not be the best implementation, and it may get moved (it's in the ProjectHelper class) but it works for now.

Stores

Since I made some good progress on Project objects this week, and I feel like I'm at a good point to move on, I decided to start working on the GDQS Store object.  Just like the Project is an extension of the GeoServer Workspace, the Store will also be a GeoServer object with GDQS specific metadata. I'm leaving off this week creating Info and New pages for the object.

Geospatial Data Quality System - QGIS integration, git tickets, and Project Info Table

Here is the recap of the work I completed on the GDQS project for this week

QGIS Integration

I'm excited to announce that I have my first integration working - launching QGIS from GeoServer.  This system needs and editor application to review the data that will be delivered to judge and make note of data quality.  Besides, what is a GIS system without a good editor application?  I've worked with a few of the OSS GIS editor applications, but I've had the best experience with QGIS.  An advantage of QGIS is that the "project" file (.qgs) is XML, which I can generate from GeoServer and stream to the user.  If QGIS is installed, the .qgs extension will be recognized by the machine and launch the application.

How was this done?  I created Wicket DownloadLink button and added it to the ProjectInfo page.  When the user clicks the button, a method will read the GDQS project information from the Data directory metadata, write the XML to a .qgs file and deliver it to the user:

I have my browser set to "always open files of type" QGIS, so the application is launched automatically after download.  Aide from the opening tag, the only info in the .qgs file is the Project name ("title" tag):

In the future, it will get populated with Layers and other info for the user.

Github account

I also now have a way to create tickets to track work and a repo to check in code.  Definitely a big step forward for my software development lifecycle:

Project Info Table

The one task that was started this week, but not finished is the Project Info table.  I would like to pull all of the Project object metadata and build a table, but only finished the work to get the table and it's header onto the page:

That is where I will start next week!

Geospatial Data Quality System - Project Pages

Geospatial Data Quality System

It was another good week for GDQS, expanding the Project object infrastructure.

I had already created a Project list page, but it was just a clone of the Workspace list page, loading all of the Workspace objects in the Catalog.  When I defined the Project object, I added a property to the Metadata Map to indicate that it is a GDQS project.  One of the parameters for the "GeoServerTablePanel" constructor is a WorkspaceProvider object, which collects all of the Workspaces in the Data Directory:

Looking at the WorkspaceProvider code, I could not figure out a good way to exclude objects so I extended the class.  The only change I had to make was overriding the getItems()  method to exclude the objects with "GDQSProject" set to "true":

Now, when the GDQS ProjectListPage loads, only GDQS projects are shown:

Aside from the ProjectProvider class, a ProjectInfo page was added, but it isn't very exciting at this point, so I'm not going to show it yet.

Ant Build

The other task I completed this week was to work on the development environment.  I created an Ant script that would copy the stop Tomcat, copy the jar, and then start Tomcat:

The next step for the build would be to run the Maven Install from Ant, or incorporate the Ant script into the pom.xml.

Geospatial Data Quality System - "Project" object definition

Geospatial Data Quality System

Hello all - it was a good week for the GDQS project, not without some difficulties, but it ended with a breakthrough.  The bad part - something broke with my out-of-the-box GeoServer codebase in the repository.  Out of nowhere, there was a compilation error and there was a dependency missing, so I pulled that code from git and ran the install.  Since the goal is to only extend GeoServer and not customize it, that code is only there for reference.  It's all back to normal now and progress is moving forward.

"Project" Object definition

This was the highlight of the week, I looked at the system layout I had sketched last week to decide what would need to be done next.  I had already decided that GeoServer was going to be the nucleus of the enterprise and work would start there:

What's going to tie all of it together?  There needs to be a context for the work, and at the top level there should be a "Project".  The GeoServer Workspace looked like an attractive option, but maybe a little too simple.  The definition of a Workspace in the Data Directory looks like this:

That's as simple as it gets, a name and an ID.  In code, the WorkSpaceInfo class is used to interact with WorkSpace objects.  The WorkSpace interface includes a MetadataMap property:

The comment says it all and this is exactly what I will do to create my "Project" subtype, add an item to the Metadata Map that identifies the object as a Project.

Project Pages

Obviously users will need a way to work with projects, a table to view the projects on the system and a way to create new projects.  For this, I created 2 new pages - the ProjectListPage and the ProjectNewPage.  The ProjectListPage looks a lot like the OOTB Workspace Page, a list of the WorkSpaces on the system:

Right now, it shows all of the WorkSpaces on the system, but with the links to non-GDQS Workspaces disabled.  In the future, those Workspaces will be filtered out from the table results.

Users will need a way to create new projects, so I created a new GDQS project page:

The page is really just a clone of the new WorkSpace page, but with some custom labels.  The code that creates the Project includes setting the "GDQSProject" property in the metadata:

So that's this week's summary.  The plan is to dive one level deeper and setup GeoServer Stores, or maybe something else for next week.

Geospatial Data Quality System - System sketch

Geospatial Data Quality System

First, I decided to change the name of my enterprise application project, I'm replacing "GIS" with "Geospatial".  GIS sounded too focused, and geospatial opens it up to include Remote Sensing, GIS, LBS, etc.

I also did some work toward the initial functional requirements, I sketched out the system design:

Right now, I will be working on the GeoServer part, creating a Project object.

The GIS Data Quality Project

Recently I decided to build my own GIS system by integrating existing software.  It's actually not a new idea, I've wanted to do it for years and now is the right time.  I've used a lot of GIS software, both Open Source (OSS) and Commercial Off The Shelf (COTS) for various applications.  I viewed the software available to me as a big toolbox that I could reach into whenever I needed to get a job done.  I filled the role of GIS "handyman" for many years in aerial mapping, but never had the opportunity to build a system.

For the past 5 years, I've been a developer and administrator on a n-tier enterprise application at a manufacturing company, which has been a radical departure from my past in GIS.  The approach to system development and data management is very rigorous and disciplined at my current company, and I would like to take what I've learned and apply it to a GIS system project.

So what are the objectives?

1. Combine the experience I outlined in the 2 paragraphs above (GIS + Enterprise Application Development)
2. Learn about System Design and the System Design Process
3. Integrate software components into a system that facilitates a workflow

There are a lot of great resources online to guide me along the process, including system design documents, development tutorials, and the manuals for the software that will be used.  The application I'm choosing is a GIS data quality evaluation system, I'm going to call it the 

GIS Data Quality System

So the adventure begins...