GDQS back on!

I took a break from my GDQS Geoserver extension project, but I'm back at it.  I've actually been working on it for 2 months, but too absorbed with it to write anything.  The thing that made me come back was buying the Activiti in Action book.  It's a comprehensive overview of Activiti, including definitions of Business Process Management (BPM), using Activiti, developing with Activiti, and a lot more.  Everything in the book can be learned from the Actitivi site, but the code examples and the corresponding explanations are great.  I've been playing around with Activiti for a couple of years, but the book is making me a pro. That also changed the development of the project.

When I left off last Spring, the code architecture for the "Project" object had been build and I was moving on to the next level of the Geoserver data structure - the Store.  The problem was the Store was not needed and, frankly, not that interesting at the moment.  I'll come back to the Store object later.

What I decided would be more interesting, is integrating another system - the BPM system in this case.  It's super easy to get up and running with Activit, just deploy the war file in Tomcat and Hello Activiti World!

The startup scripts for Activiti include example processes, users and also configured out of the box to use an in memory H2 database to get you started.  There are actually 2 war files with the download - the Activiti Explorer web app and the REST API for the Activiti Engine.  This is where I will be focusing my efforts in the upcoming weeks:

GDQS - time off from the project

Geospatial Data Quality System

Since I had relatives visiting this week, I decided to take the week off from this project.  I knew I would be getting behind on work at my day job, so I spent a little time getting caught up on that.

I did have a little free time to do some digitizing in QGIS.  I'm going to need some data to QC once the project gets to the point of proving the capabilities.  It also gave me some time to explore the Web Map Services (WMS) that are available.  A search for Imagery WMS, gave me many options, but I chose the USGS EROS high res imagery (1 foot orthos).  I create a new QGIS WMS layer, picked a scale to collect from, and started digitizing:

This will give me a good dataset to QC later one.  Man, am I out of practice with digitizing!

Store achitecture

Tomorrow, I will continue building the Store architecture.

Geospatial Data Quality System - Project Info Table, GeoServer Catalog reloading, and Stores

Project Info Table - easily solved

When I left off last week, I was struggling with trying to display the Project metadata in a table on the Project Info Page.  Well, that was a case of getting frustrated and not thinking clearly and deciding I would tackle it during the next week.  I had a little free time to get on the computer Monday night, so I thought I would give it a shot, and was able to get the table display working within a few minutes.  Solving the problem was a matter of taking a break.

It's not the best implementation at the moment, since I need to add a new table row to the html template for each new attribute I add to the Project object (and it must match the attribute key):

"ProjectStatus" attribute added to the Project object

It is relatively simple on the Java side, and requires no code changes to display the new attribute value.  It simply iterates through the MetadataMap and creates the wicket Label objects to display.

I would like to simplify this further but this works for now.  In the future, I would like to have the whole table generated in code, not having to add anything to the html page.

GeoServer Catalog Reloading

As I was working on the MetadataMap table display, I created a couple of new Projects and noticed that those did not immediately display on the PorjectList page.  It would require restarting Tomcat to see any new projects on the ProjectList page, while I could see the new Project on the Workspace list page.  I figured there must be a way to refresh the cache of loaded project, so I dug through the code.  This is what I found:

That may not be the best implementation, and it may get moved (it's in the ProjectHelper class) but it works for now.

Stores

Since I made some good progress on Project objects this week, and I feel like I'm at a good point to move on, I decided to start working on the GDQS Store object.  Just like the Project is an extension of the GeoServer Workspace, the Store will also be a GeoServer object with GDQS specific metadata. I'm leaving off this week creating Info and New pages for the object.

Geospatial Data Quality System - QGIS integration, git tickets, and Project Info Table

Here is the recap of the work I completed on the GDQS project for this week

QGIS Integration

I'm excited to announce that I have my first integration working - launching QGIS from GeoServer.  This system needs and editor application to review the data that will be delivered to judge and make note of data quality.  Besides, what is a GIS system without a good editor application?  I've worked with a few of the OSS GIS editor applications, but I've had the best experience with QGIS.  An advantage of QGIS is that the "project" file (.qgs) is XML, which I can generate from GeoServer and stream to the user.  If QGIS is installed, the .qgs extension will be recognized by the machine and launch the application.

How was this done?  I created Wicket DownloadLink button and added it to the ProjectInfo page.  When the user clicks the button, a method will read the GDQS project information from the Data directory metadata, write the XML to a .qgs file and deliver it to the user:

I have my browser set to "always open files of type" QGIS, so the application is launched automatically after download.  Aide from the opening tag, the only info in the .qgs file is the Project name ("title" tag):

In the future, it will get populated with Layers and other info for the user.

Github account

I also now have a way to create tickets to track work and a repo to check in code.  Definitely a big step forward for my software development lifecycle:

Project Info Table

The one task that was started this week, but not finished is the Project Info table.  I would like to pull all of the Project object metadata and build a table, but only finished the work to get the table and it's header onto the page:

That is where I will start next week!

Geospatial Data Quality System - Project Pages

Geospatial Data Quality System

It was another good week for GDQS, expanding the Project object infrastructure.

I had already created a Project list page, but it was just a clone of the Workspace list page, loading all of the Workspace objects in the Catalog.  When I defined the Project object, I added a property to the Metadata Map to indicate that it is a GDQS project.  One of the parameters for the "GeoServerTablePanel" constructor is a WorkspaceProvider object, which collects all of the Workspaces in the Data Directory:

Looking at the WorkspaceProvider code, I could not figure out a good way to exclude objects so I extended the class.  The only change I had to make was overriding the getItems()  method to exclude the objects with "GDQSProject" set to "true":

Now, when the GDQS ProjectListPage loads, only GDQS projects are shown:

Aside from the ProjectProvider class, a ProjectInfo page was added, but it isn't very exciting at this point, so I'm not going to show it yet.

Ant Build

The other task I completed this week was to work on the development environment.  I created an Ant script that would copy the stop Tomcat, copy the jar, and then start Tomcat:

The next step for the build would be to run the Maven Install from Ant, or incorporate the Ant script into the pom.xml.

Geospatial Data Quality System - "Project" object definition

Geospatial Data Quality System

Hello all - it was a good week for the GDQS project, not without some difficulties, but it ended with a breakthrough.  The bad part - something broke with my out-of-the-box GeoServer codebase in the repository.  Out of nowhere, there was a compilation error and there was a dependency missing, so I pulled that code from git and ran the install.  Since the goal is to only extend GeoServer and not customize it, that code is only there for reference.  It's all back to normal now and progress is moving forward.

"Project" Object definition

This was the highlight of the week, I looked at the system layout I had sketched last week to decide what would need to be done next.  I had already decided that GeoServer was going to be the nucleus of the enterprise and work would start there:

What's going to tie all of it together?  There needs to be a context for the work, and at the top level there should be a "Project".  The GeoServer Workspace looked like an attractive option, but maybe a little too simple.  The definition of a Workspace in the Data Directory looks like this:

That's as simple as it gets, a name and an ID.  In code, the WorkSpaceInfo class is used to interact with WorkSpace objects.  The WorkSpace interface includes a MetadataMap property:

The comment says it all and this is exactly what I will do to create my "Project" subtype, add an item to the Metadata Map that identifies the object as a Project.

Project Pages

Obviously users will need a way to work with projects, a table to view the projects on the system and a way to create new projects.  For this, I created 2 new pages - the ProjectListPage and the ProjectNewPage.  The ProjectListPage looks a lot like the OOTB Workspace Page, a list of the WorkSpaces on the system:

Right now, it shows all of the WorkSpaces on the system, but with the links to non-GDQS Workspaces disabled.  In the future, those Workspaces will be filtered out from the table results.

Users will need a way to create new projects, so I created a new GDQS project page:

The page is really just a clone of the new WorkSpace page, but with some custom labels.  The code that creates the Project includes setting the "GDQSProject" property in the metadata:

So that's this week's summary.  The plan is to dive one level deeper and setup GeoServer Stores, or maybe something else for next week.

Geospatial Data Quality System - System sketch

Geospatial Data Quality System

First, I decided to change the name of my enterprise application project, I'm replacing "GIS" with "Geospatial".  GIS sounded too focused, and geospatial opens it up to include Remote Sensing, GIS, LBS, etc.

I also did some work toward the initial functional requirements, I sketched out the system design:

Right now, I will be working on the GeoServer part, creating a Project object.