Waiting for QGIS 2.2 – Gradient Fills

One of the big features I worked on for QGIS 2.2 is gradient fill symbols for polygons. In my view QGIS’ symbol support is one of its biggest strengths — the versatility of its symbol layers coupled with the powerful data defined properties support allows for so many effects which just aren’t possible in other GIS packages. Gradient fill support is a nice addition to these features and should help make QGIS even more attractive to cartographers. In this post I’m going to give a quick run through of how gradient fills work in QGIS, and some of the options available for tweaking them.

Gradient fills are enabled through the Style tab in the properties for a vector layer. The default fill for a polygon in QGIS is “Simple fill”, so to switch a layer to a gradient fill you first need to select the “Simple fill” layer, then change the “Symbol layer type” dropdown  to “Gradient fill”:

Gradient Fill type

As you can see, there’s a lot of options in QGIS which can be tweaked for gradient fills. I’ll run through each of them now and explain a little bit about how each one can be used.

Colour modes

QGIS supports two different types of colour modes for gradient fills. The first is a simple “Two color” gradient, where the colour smoothly blends from the first colour to the second. The second mode, “Color ramp” allows you to use any of the standard or user-defined QGIS colour ramps, which can consist of multiple colour stops:

Colour options in gradient fills

Colour options in gradient fills

So, when would you use these options? Well, any time you need more than two colours or need to tweak the position of any of the colours in the gradient you’ll have to use a colour ramp.  If instead you’re just wanting a quick-and-easy gradient then the two colour option might be more suitable.

One last important distinction is that the colours in a two colour gradient can be set using a data defined expression:

Data defined gradient colours

Data defined gradient colours. Please try to use them more tastefully then this!

Gradient types

The next option for gradient fills is rather self-explanatory: gradient types. QGIS supports linear, radial and conical gradients:

Gradient types

Coordinate modes

The coordinate mode option is a little trickier to explain. The default setting, “Object“, will cause the gradient to be drawn entirely within each separate feature. You can see in the example below that every lake feature is coloured with a gradient which starts with light blue in the top left and darkens to a deeper blue in the bottom right. This gradient fill is repeated for all the lake features:

Gradient object coordinate mode

The “Object” coordinate mode for gradient fills

In contrast, the “Viewport” coordinate mode causes the gradient to be drawn across the entire current view of the map. So only the lakes in the top left of the map are drawn with the light blue colour, and the lakes in the bottom right with the deeper blue:

Gradient "viewport" coordinate mode

The “Viewport” coordinate mode for gradient fills

The choice of coordinate mode will depend entirely on your cartographic desires for your map!

Reference points

QGIS gradient fills allow the setting of two “reference points“. These points control where the gradient fill begins and ends. It’s easiest to visualise how these work by imagining a square defined by the points (0, 0) in the top left and (1, 1) in the bottom right. The two reference points fall somewhere within this square. So, the default reference points of (0.5, 0) and (0.5, 1.0) represent points mid way along the top edge and and the bottom edge, respectively.

Now imagine that this square forms the bounding box for the feature being drawn (or the current map window, if in “viewport” coordinate mode). The default reference points mean that the gradient will be drawn from the middle of the top edge to middle of the bottom edge of the feature. Reference points of (0, 0) and (1, 1) would mean the gradient is drawn from the top-left to the bottom-right. Similarly, reference points of (0.5, 0.5) to (1.0, 1.0) would draw a gradient from the middle of the feature to the bottom right (good for radial gradients).

Example gradient reference points

There’s also the option to set either of the reference points as the feature centroid, which again can come in handy for radial or conical gradient types.

Gradient spread

If you’ve got your head around the reference points concept, then the next setting for gradient fills affects how the gradients spread. This takes effect whenever a gradient starts or ends before the bounds of the feature. The default setting of “pad” means that the gradient will simple “pad” out any extra space with the start or end gradient colour:

Gradient "pad" spread

“Pad” spread – notice how the darker blue is stretched across the right side of each feature

Repeat” mode will tile the gradient across the feature:

Gradient "repeat" spread

“Repeat” spread

Finally, “reflect” mode will draw a reflected version of the gradient to fill up any extra space:

"Reflect" spread

“Reflect” spread

Angle

Last of all, there’s a simple “angle” parameter, which allows you to rotate the entire gradient fill. This option is included mostly for use with data defined symbols, since a similar effect can be achieved by changing the gradient reference points. Amongst other effects, this is useful for achieving a “sun glint” on water, where each gradient is drawn in a random direction (more on this in a later blog post):

Data defined gradient angles

Random data defined gradient angles

This leads me into my final note… all of these properties can be data defined! So you could have a column in your data controlling whether each feature is drawn with a radial or linear gradient, or whether the gradient in a given feature should be drawn at a specific angle, or that the gradient in a feature should start at the centroid and end at the top right of the feature!

I’m excited to see what the QGIS user community is able to create using this new gradient fill feature when 2.2 is released. If you’ve already had a chance to play with the dev version of 2.2 and have something to show off, make sure you submit your map to the Flickr QGIS Map Showcase!

Tagged , , , ,

Waiting for QGIS 2.2 – Composer Improvements (part 2)

Following on from Part 1, here’s some more recent usability improvements which have been made to QGIS’ map composer:

  • Multiple items can now be selected by drawing a box around them. Holding Shift while drawing a box adds to the current selection, holding Ctrl will subtract from the current selection. In addition, holding Alt while drawing the selection box changes from an “intersects” style selection mode to a “within” selection mode. This was my major frustration with the composer in earlier versions of QGIS, and combined with the earlier fixes for multi-item drag and resize it substantially improves the design workflow.
Selecting multiple items with a mouse drag

Selecting multiple items with a mouse drag

  • Grid lines are always drawn on top of compositions
  • There’s new menu options for showing the grid and snapping to grid, and for showing/snapping/smart guides. All these have convenient keyboard shortcuts so you can quickly switch on or off display of the guides and grid or temporarily switch on or off snapping while you work. There’s also a new menu item for clearing all guides from a composition.
Grid and guide controls in the view menu

Grid and guide controls in the view menu

  • The composer window now has a status bar which shows the exact cursor position and details about resizing/moving items
The new composer status bar

The new composer status bar

  • A new mouse-based zoom tool has been added to the composer, which allows you to click and drag to specify an area to zoom to. You can also switch to this new tool at any time by holding Ctrl and space. Holding Shift while using the zoom tool switches to zooming out.

That’s it for now, but there’s loads more to come for 2.2!

Tagged , , , ,

Waiting for QGIS 2.2 – Composer Improvements (part 1)

In the spirit of depesz’s “Waiting for Postgresql 9.#” series (and BostonGIS’ follow up Waiting for PostGIS 2.#), here’s the first in a new series of “Waiting for QGIS 2.2” posts…

As I mentioned in my last post, one of my major goals for QGIS 2.2 is to make the print composer behave more like a professional DTP package. A big part of this is making sure QGIS respects all the expected standard behaviour for graphic design programs. While the print composer made huge advances toward this goal in version 2.0, there’s still further we can go.

So, let’s start with a list of all the new usability features the print composer has already gained for QGIS 2.2:

  • Selections of multiple items can all be moved and resized together (previously only one item in the selection would be altered)
  • Mouse handles are always drawn on the top of the composition – in earlier versions mouse handles could be hidden by items sitting higher in the composition
  • Holding shift while resizing items locks their aspect ratio
  • Holding control while resizing items causes them to resize from their centre
  • Holding shift while moving items locks the movement to the horizontal or vertical
  • Holding control while moving items temporarily disables any grid or guide-based snapping
  • Items can be removed from a selection by shift-clicking them
  • Control-clicking an item causes the next item below it to be selected – this allows you to select items which are hidden behind other items
  • Holding shift while moving items using the cursor keys results in a large item movement
  • There’s a new “Edit” menu in the composer, with options like
    • Select All/Select None and Invert Selection
    • Select Next Item Above and Select Next Item Below
    • Lock Selected Items and Unlock All
The new composer "Edit" menu

The new composer “Edit” menu

Additionally, there’s a bunch of pull requests which haven’t yet been merged to master, but are ready to go, including:

  • Adding a dedicated “pan” tool for dragging around compositions
  • Compositions can be panned at any time by holding the space key or middle mouse button
  • Compositions can be zoomed by using the mouse wheel
Composer pan tool

Composer pan tool

Some of these are simple little changes which don’t sound like much, but it’s not until you’re forced to work without something like shift-resizing to lock the aspect ratio that you realise how often you use it and rely on it!

Tagged , , , ,

Sneak Peak

While it’s really exciting that the release of QGIS 2.0 is just around the corner, I’m finding it just as exciting that work is already under way on version 2.1! Some great new features have already landed and I can’t wait to see what else is planned. Personally, I’ve got plans for a raft of improvements relating to print composers with the goal of making the print composer behave as much like a full-blown DTP package as possible. In the meantime, here’s a quick teaser of a feature I’ve been working on that will hopefully be landing in master soon:

A sneak peak...

Tagged , , , ,

VicMap data now freely available!

This is pretty huge – it seems the majority of the VicMap dataset is now freely available on data.vic.gov.au! I haven’t found an official release announcement but it includes the entire address, roads, property, public transport and administrative boundary data, plus much more. In fact, the only missing data I’ve found so far is the features of interest dataset. I’m not sure why that one didn’t make the cut.

One of the best things about this is that the data has been released under a very permissive CC-BY 3.0 license. With any luck, we’ll be able to merge this data with Victoria’s OpenStreetMap data. (There’s just a small licensing issue which needs clarifying).

(A bit of advice if you are trying to export large amounts of this data – the quickest format to export in is using “ESRI ArcSDE Shape Export” with “Geographicals on GDA-94” projection. Other formats can take a long time to process before they’re available for download.)

Tagged , ,

Detecting key states in MapBasic

Here’s another neat little MapBasic function I’ve been using recently. I needed a way of detecting whether a key on the keyboard was currently pressed or not. My intention was to show a hidden debugging dialog if a user shift-clicked on a specific button control, but there’s no in-built functions in MapBasic for detecting key states. Fortunately it’s possible to hook into the Windows User32.dll to use the GetAsyncKeyState call. Here’s how this all works in MapBasic…

First, we’ve got to define the GetASyncKeyState call and a bunch of related constants which map keys to a numeric value. Copy and paste the contents below to a .def file (or download a pre-made version later in this post).

asynckeys.def:

Declare Function GetAsyncKeyState Lib "User32.dll" Alias "GetAsyncKeyState" (ByVal vKey As Integer) As Integer
DEFINE vbKeyShift 16
DEFINE vbKey1 49
DEFINE vbKey2 50
DEFINE vbKey3 51
DEFINE vbKey4 52
DEFINE vbKey5 53
DEFINE vbKey6 54
DEFINE vbKey7 55
DEFINE vbKey8 56
DEFINE vbKey9 57
DEFINE vbKey0 48
DEFINE vbKeyBack 8
DEFINE vbKeyTab 9
DEFINE vbKeyReturn 13
DEFINE vbKeyControl 17
DEFINE vbKeyMenu 18
DEFINE vbKeyPause 19
DEFINE vbKeyEscape 27
DEFINE vbKeySpace 32
DEFINE vbKeyEnd 35
DEFINE vbKeyHome 36
DEFINE vbKeyLeft 37
DEFINE vbKeyRight 39
DEFINE vbKeyUp 38
DEFINE vbKeyDown 40
DEFINE vbKeyInsert 45
DEFINE vbKeyDelete 46
DEFINE vbKeyMultiply 106
DEFINE vbKeyDivide 111
DEFINE vbKeyAdd 107
DEFINE vbKeySubtract 109
DEFINE vbKeyDecimal 110
DEFINE vbKeyF1 112
DEFINE vbKeyF2 113
DEFINE vbKeyF3 114
DEFINE vbKeyF4 115
DEFINE vbKeyF5 116
DEFINE vbKeyF6 117
DEFINE vbKeyF7 118
DEFINE vbKeyF8 119
DEFINE vbKeyF9 120
DEFINE vbKeyF10 121
DEFINE vbKeyF11 122
DEFINE vbKeyF12 123
DEFINE vbKeyNumlock 144
DEFINE vbKeyScrollLock 145
DEFINE vbKeySnapshot 44
DEFINE vbKeyPageUp 33
DEFINE vbKeyPageDown 34
DEFINE vbKeyNumpad1 97
DEFINE vbKeyNumpad2 98
DEFINE vbKeyNumpad3 99
DEFINE vbKeyNumpad4 100
DEFINE vbKeyNumpad5 101
DEFINE vbKeyNumpad6 102
DEFINE vbKeyNumpad7 103
DEFINE vbKeyNumpad8 104
DEFINE vbKeyNumpad9 105
DEFINE vbKeyNumpad0 96
DEFINE vbKeyPressed -32767
DEFINE vbKeyWasPressed 1

Now, you can test for the state of any key by calling getASyncKeyState( vKey ), where vKey is an integer value corresponding to the key you want to test. So getASyncKeyState( vbKeyControl ) will test the state of the control key. The value returned will reflect whether the key is currently pressed (vbKeyPressed), or has been pressed since the last call to the function (vbKeyWasPressed). Easy! Here’s a little sample program to demonstrate how this all works:

Include "asynckeys.def"
Declare Sub Main
Declare Sub KeyCheck

Sub KeyCheck
    ' Store the pressed state of the SHIFT key
    Dim iState As Integer
    iState = getASyncKeyState( vbKeyShift )

    ' Check what the state was
    Do Case iState
        Case vbKeyPressed
            Print "SHIFT is being pressed!"
        Case vbKeyWasPressed
            Print "SHIFT has been pressed since last check"
        Case Else
            Print "Nothing to report..."
    End Case

    ' Keep the dialog around
    Dialog Preserve
End Sub

Sub Main
    Dialog
        Title "ASync Key State"
        Control OKButton
            Title "Check"
            Calling KeyCheck
        Control CancelButton
            Title "Quit"
End Sub

Both the asynckeys.def definition file and the sample program are contained in this archive.

Tagged , , , , ,

VicRoads and “Pseudo AMG”

Recently I was attempting to import some data sourced from VicRoads, but had a huge struggle trying to find the correct coordinate system to plot it in. The data had columns labelled amg_x and amg_y, so naturally I assumed this referred to either AMG66 zone 54 or 55. Neither of these fit though – the points were plotting about 200km out from their correct location.

It turns out VicRoads “helpfully” uses their own coordinate system, Pseudo AMG (described on pages 18 & 19 in this document). If you’re trying to import this data into a modern GIS, this coordinate system can be described by the proj string:

+proj=tmerc +lat_0=0 +lon_0=145 +k=1 +x_0=500000 +y_0=10000000 +ellps=WGS84
+towgs84=-117.808,-51.536,137.784,0.303,0.446,0.234,-0.29 +units=m +no_defs

With any luck this may save someone else the time I spent trying to mangle this data into shape. Thanks VicRoads… ThicRoads.

Tagged , ,

Regular expressions in MapBasic

I’m going to take a break from all my recent QGIS posts to talk about some MapBasic news… I’m proud to announce the release of MbRegEx, an open-source library for using regular expressions in MapBasic scripts!

If you’re not familiar with regular expressions, they’re an extremely powerful tool for string manipulation. They can be somewhat daunting at first, but with a bit of practice they’ll open up all kinds of string processing which would otherwise be extremely convoluted or impossible.

Up until now there’s been no way of using the beauty of regular expressions within MapInfo. Now, with MbRegEx, all their mighty power can be fully utilised within your MapBasic scripts!

Included Functions

The library contains 5 functions for use in MapBasic scripts:

RegExTest(string, regex): returns true or false depending on whether string contains the specified regular expression. For example, RegExTest( “ABC123”, “[A-Z]{3}\d{3}” ) returns true, RegExTest( “AB12”, “[A-Z]{3}\d{3}” ) returns false. This function really comes in handy when selecting records, eg:

SELECT * FROM address WHERE RegExTest( street, "MAIN (ST|RD)( [NESW])?$")

to select all addresses where the street is either “MAIN ST” or “MAIN RD” and which may have an optional N/E/S/W suffix.

RegExMatch(string, regex): returns the first part of string which matches the specified regular expression. Great for quickly extracting specific parts of a string. Let’s say we’ve got an address field, and we’d like to grab just the street number from it. We could use the regular expression “^(\d+[A-Z]*)”, which will match any leading numbers with optional letter suffixes. So RegExMatch( “12A Main St”,  “^(\d+[A-Z]*)” ) returns “12A”,  whereas RegExMatch( “Upper Main St”,  “^(\d+[A-Z]*)”) will return an empty string. Trying to achieve this same task using built-in MapBasic functions alone would be extremely tedious and error-prone!

RegExMatchAll(string, regex, array) and RegExMatchMultiple(string, regex, array) will fill array with all the parts of string which pass the specified regular expression. RegExMatchAll is used when you have just one capturing group in your regular expression, and RegExMatchMultiple is used for multiple capturing groups.

This is probably best demonstrated with some examples:

RegExMatchAll( "12A Main St", "\b(\w+?)\b", sMatches )

will fill the sMatches array with all the individual words from “12A Main St”. So sMatches(1) = “12A”, sMatches(2) = “Main”, etc.

RegExMatchMultiple( "10.5.2", "(\d+)\.(\d+)\.(\d+)", sMatches )

fills sMatches with the results of the three capturing groups (the bracketed parts) of the regular expression. Thus sMatches(1) = “10”, sMatches(2) = “5” and sMatches(3) = “2”.

Again, these two functions are great for splitting a string into its component parts or doing something fancy like extracting all the phone numbers from a paragraph of text.

Lastly, RegExReplace(string, regex, replacement, destination) replaces the parts of string which match the regular expression with replacement, and stores the result in destination. One handy use for this is removing invalid characters from a string, such as

RegExReplace("T/:e@st!i~n&g#1,^2}3", "[^A-Za-z0-9]", "_", sDest)

This results in sDest = “T__e_st_i_n_g_1__2_3”.

Getting Started

There’s a few extra examples and full instructions for using these functions in the “test.mb” file included with MbRegEx. You can use these as a starting point for your own MapBasic scripts. If regular expressions are new to you, I recommend regular-expressions.info as a great resource together with an online regex visualiser such as debuggex.

Also, a quick warning – MbRegEx does no validation or testing of your regular expressions, so be careful with badly formatted expressions as they’ll cause MapInfo to crash. If in doubt, run the expression through an online tool to validate it first.

Downloads

The full source code is available on GitHub, under a public domain license. If you’re looking for the easiest way to get started, just download this zip which contains both the MbRegEx DLL and a sample MapBasic program demonstrating all the different regular expression functions which are available in the library.

End note – building the MbRegEx DLL file

I’ve included a CodeBlocks project file with the source, but building the MbRegEx.dll file can be difficult – you’ll need to first compile the boost libraries using mingw on your system. Then, make sure that in the CodeBlocks project build options, under Linker settings you have your “stage\lib\libboost_regex-mgw46-1_52.a” (or similar) correctly linked. Also, under Search directories add your boost folder to the Compiler tab. 

Tagged , , , ,

Building QGIS on Ubuntu 13.04

It seems upgrading to Ubuntu 13.04 breaks a few things in the QGIS build process. Here’s some quick workarounds I’ve found to get it building again.

Firstly, Ubuntu defaults to Qt 5, causing the error:

CMake Error at /usr/share/cmake-2.8/Modules/FindQt4.cmake:1216 (message):

 Found unsuitable Qt version "5.0.1" from /usr/bin/qmake, this code requires
 Qt 4.x

This can be fixed by switching the system default back to Qt 4, so that  qmake refers to the Qt 4 version. The “qt4-default” package handles this switch for you, so just run:

sudo apt-get install qt4-default

Update: A better solution was found by Pierre and Pvanb and is described here. Basically it involves changing the QMAKE_EXECUTABLE option from /usr/bin/qmake to /usr/bin/qmake-qt4.

Next, I was getting the error:

make[2]: *** No rule to make target `/usr/lib/libpython2.7.so', needed by `output/lib/libqgispython.so.1.9.0'. Stop.

I’m not sure if this is the correct solution, but setting up some links and re-running ldconfig gets around this:

sudo ln -sf /usr/lib/x86_64-linux-gnu/libpython2.7.so.1 /usr/lib/libpython2.7.so.1
sudo ln -sf /usr/lib/x86_64-linux-gnu/libpython2.7.so /usr/lib/libpython2.7.so
sudo ldconfig

Update: a better solution is to change the PYTHON_LIBRARY option from /usr/lib/libpython2.7.so to /usr/lib/x86_64-linux-gnu/libpython2.7.so for 64 bit environments, or /usr/lib/libpython2.7.so to /usr/lib/i386-linux-gnu/libpython2.7.so for 32 bit environments

These two changes were enough to get QGIS building again. If you’ve got a better solution for these errors, let me know…

Tagged , , ,

A neat trick in QGIS 2.0 – images in atlas prints

Here’s a cool trick which you can do in QGIS 2.0. It builds on two new features introduced in version 2.0 – atlas prints and html labels.

Atlas prints (previously available as a plugin, now integrated into QGIS core) were developed by Oslandia, and allow you to create data driven map layouts from a specified coverage layer. You can read more about them here.

Another new feature in QGIS 2.0 is the ability to render composer labels as html (courtesy of Olivier Dalang). This allows all kinds of fantastic effects, such as formatting text in the middle of a label (using <b>, <i>, and <font> tags) or creating labels which contain HTML tables. You can even use CSS stylesheets and rules to format the HTML! I’ve been told JavaScript also works inside the labels, but I’ve yet to try this out.

You can combine these two new features for some great tricks. Let’s say we’d like to create a set of maps of local councils, and we want each map to have a watermarked logo of the council on it. For this example I’ve created a basic basemap of Victorian councils, and I’ve downloaded all the council logos (in a variety of formats) to a local folder. Next, I’ll add an extra column to the council layer containing the name of the logo image:

adding_logo_column

Adding a logo column to the table

Then, we’ll throw together a simple composition containing the map and set it up as an atlas print:

setting_up_atlas

Generating an atlas

Now for the fun bit. I’ll add a label item to the composer, set it to “Render as HTML”, and insert some specially-crafted html:

The magic HTML label...

The magic HTML label…

For copy/paste purposes here’s the label contents again:

<style>
* {margin: 0px; padding: 0px}
</style>
<img src="file:///home/nyall/GIS/council_logos/[% "logo" %]"
style="width: auto; height: 100%; display: block; margin: 0px 0px 0px auto;" />

There’s a couple of things to note here. First, the magic happens in the image’s src attribute (“file:///home/nyall/GIS/council_logos/[% “logo” %]”). When the composer is exported, QGIS will replace the [% “logo” %] part with the contents of the logo field for each row in the councils table. This means the image source will then point to the local copy of the council’s logo, eg “file:///home/nyall/GIS/council_logos/glen_eira.jpg” for the first row.

Secondly, I’ve styled the image with the css:

width: auto; height: 100%; display: block;

This allows the image to resize to 100% of the height of the label while maintaining its correct aspect ratio. I’ve also added the rule

margin: 0px 0px 0px auto;

to force the image to right align within the html label. This ensures that all the watermarked logos will appear in a consistent size and position for each map.

Lastly, I’ll remove the label’s frame and background by unchecking these options, then set its transparency to 80% under the new “Rendering” section:

Yet another new feature in QGIS 2.0...

Yet another new feature in QGIS 2.0…

Ok, we’re all done. Now, when I select Composer -> “Export as Image”, we’ll get a lovely set of council maps complete with watermarked council logo!

A watermarked atlas!

A watermarked atlas!

There we go — all ready for print, with no manipulation in external programs required at all!

Bonus post-credits section

Here’s a kicker — the linked images don’t need to be local. That means you can even piggy-back off an existing web service to generate an image on the fly! Let’s say you were asked to add QR codes to your maps to link directly to the council website. All it takes is adding a new column to the table, then modifying the image src to read:

src="http://qrfree.kaywa.com/?l=1&s=8&d=[% "url" %]"

Now when we export the maps we’ll also get a QR image generated on the fly and inserted into the layout!

Complete with dynamically generated QR code!

Complete with dynamically generated QR code!

Combining HTML labels and atlas prints in this way is extremely powerful. This example is just touching the tip of the iceberg – I’m keen to see what the community can do with this when QGIS 2.0 is released!

Tagged , , , , , , ,