The Topobase Insiders

Topobase help online

Great news for all the Topobase users and 3rd party developers. Autodesk just published the full Autodesk Topobase 2010 help on www.autodesk.com/topobase-help. It is now available online and accessible from everywhere you are.

Here you find the direct link to each of the user guides:

Administrator Guide - Describes administrator tasks such as designing forms, creating reports, establishing rules for job management, setting up users and groups, managing workspaces, documents, configuration settings and topologies, customizing workflows, and creating and editing the data model.
 
Feature Rules Reference - Provides a brief overview of the concept of feature rules and includes sections that describe the basic server-side and client-side rules, the rules that you can use with feature classes in the Coordinate Geometry (COGO), Template, and Plot extensions, and rules that you can associate with the specialized feature classes of the Topobase vertical application modules.

Client User Guide - Provides explanations and procedures for using Topobase Client to display, create, edit, and analyze geometry and attribute data.

Web User Guide - Provides concepts and procedures for using Topobase Web.

Electric User Guides - The North American and Central European Electric User Guides describe the data models and workflows specific to the electric industry in those regions.

Gas User Guide - The Gas User Guide describes the data models and workflows specific to the gas industry.

Land Management User Guide - The Land Management User Guide describes the data models and workflows specific to land management.

Survey User Guide - The Survey User Guide describes the data models and workflows specific to surveying.

Wastewater User Guide - The Wastewater User Guide describes the data models and workflows specific to the wastewater industry.

Water User Guide - The Water User Guide describes the data models and workflows specific to the water industry.

Developer’s Guide - Describes how to use the Topobase API and associated sample code.

Also AutoCAD Map3D and AutoCAD Civil3D have their documentation online.

Model I/O

The Topobase Administrator provides an easy way to manually create and maintain the data model for each document. However, there is a need to be able to accurately and repeatably replicate a data model, for example when moving from a development to a testing or production environment. Topobase has a very rich API for this purpose and a mechanism called structure update. This is used by vertical modules to ensure that the expected data model is present when they are added to the document. But somebody has to write the code to create the structure update.

With this use-case in mind, the open source project ModelIO comes to the rescue. It provides a way to extract a Topobase data model to an XMI file, and conversely, generate a data model based on an XMI data file - so no one has to write the code. The project also includes an FDO utility to perform the same actions on an FDO data store.

Although documentation is a little scarce at the moment, a new version targeting Topobase 2010 has been created. Give it a try.

DeepEarth with Silverlight 2.0

Here is a very nice example what is possible with Microsoft’s Silverlight 2.0 platform. The web page below shows a dynamic zooming inside a mapping control. It looks and feels similar to Google Earth but it works within the web browser. Currently supported platforms for Silverlight are Firefox, Safari and Internet Explorer on Windows, MacOS and Linux.

Give it a try: http://deepearth.soulsolutions.com.au/

 

For the web developers:
The Silverlight control which is beeing used is called DeepZoom and displays muli-scale images. Another use case where you can see this control in action is here: http://memorabilia.hardrock.com/

Spatial selection

Topobase offers several different possibilities how to filter features. Simple attributive filters, SQL based filters or geographical filters. All these filters can be managed easily and also saved for later usage.

The following video shows a couple of interesting things which can help you searching for features. First you see how to digitize a polygon in the map in order to retrieve all the point features inside this polygon. Then you can add additional conditions, in this example I add a condition to only select the landmark points inside this polygon (select Additive Filter!). Then I'm saving this filter in the Filter Manager for later usage. If you want that other users can also use this filter you can make it public.

Accessing Documents from AutoCAD Commands

Although it is easy to access the Topobase Document object from within a PlugIn such as a Workflow, if you need to access the Topobase API when your code is invoked from the AutoCAD command line or a menu item, you will need a small trick.

The idea is to implement a DocumentPlugIn that provides a static accessor to the Topobase context, like the Application and collection of Document objects – the Singleton pattern. This may take this form:

using System;

using Topobase.Forms;

namespace SingletonPlugIn

{

/// <summary>

/// Provides static access to Topobase context.

/// </summary>

public class StaticConnectionDocumentPlugin : DocumentPlugIn

{

protected static Application _Application = null;

public static Application StaticApplication

{

get { return (_Application); }

}

public static Document StaticDocument

{

get { return (_Application.Documents.Active); }

}

public override void OnInitComplete (object sender, EventArgs e)

{

_Application = Document.Application;

}

}

}

The only non-static method in the class is the OnInitComplete, which squirrels away the Application object – from which all other Topobase context items can be obtained – when the document is loaded.

You will need a TBP file or an entry in an existing TBP file to load it, something like:

<DocumentPlugIn Namespace="SingletonPlugIn" ClassName="StaticConnectionDocumentPlugin" />

Then, it is possible to get at the current document, from an AutoCAD command, using the static members, like so:

using Autodesk.AutoCAD.Runtime;

[CommandMethod ("MyCommand")]

public void MyCommand ()

{

Document document;

document = StaticConnectionDocumentPlugin.StaticDocument;

…

}

It is even possible to call a Topobase workflow, as long as you provide a WorkflowSupport object with the document. Assume you have a workflow class like this:

public class MyWorkFlow : WorkflowPlugIn

{

public void DoTheNeedful ()

{

…

}

}

Then you can call it from an AutoCAD command with code like this:

MyWorkFlow workflow;

workflow = new MyWorkFlow ();

workflow.WorkflowSupport = new WorkflowSupport (document);

workflow.DoTheNeedful ();

How to optimize your Topobase Forms

The Topobase Forms and it's functionality is one of the most used components within Topobase Client and Topobase Web. By improving its performance you can speed up your standard tasks very easily. Just read the following Do's and Don'ts and think about where you can gain some performance.

What you should do:

·         If you have repeating queries or part of queries, use a PL/SQL stored procedure to capsulate them. You can reuse it then. This has a huge impact on performance.

·         Disable controls in grid mode (table view), if you don’t need them there.

·         If you don’t need a Control (SQL Label/SQL Textbox) at all, remove it from the form and not only hide it.

·         Think about if you really need a value to be displayed right now. You can also use a “Script Programmable Control” (It’s a Button the user can click and this executed a VB.NET Script where you can use the full Topobase API) to calculate the Value on demand and display it in a Message box instead.

Sample code:

Public Overrides Sub Button_Click()

   dim myResult as string

   Me.ConnectionTools.StrValue("select name from pipe where fid_parent=" & me.LngValue("FID"))

   Me.Application.MessageBox("The pipename is:" & myResult)

End Sub

What you should not do:

·         Don’t use SQLLabels if the data is already loaded into the form. Use the”Formula Textbox” instead. If your form is WW_LINE and you have a select like this:

„Select length * 1000 from ww_line where fid = {fid}“

The column “length” is already available in memory and accessible via the API. The API can be called via the “Formula Textbox”. So add a “Formula Textbox” with this code:

if me.IsNull("length") then

    result=""  ‘If the length has a NULL Value…..

else

    result=Me.DblValue("length ") * 1000

End if

·         Don‘t use SQLLabels if you only want to lock a control. This can be done by the properties of the control.

·         Don’t use SQLLabels if you want to use an attribute of the feature in more than one tab of the form. Use the “Formula Textbox” instead.

·         Check that your SQL is fast by using Explain Plan and also check that you have indexes where necessary.

Sample for the Topobase SQL Sheet:

“ExplainPlan select * from tb_info where id=100”

Watch out for a “full table scan” (this means the indices are not used at all) if you query less than 15% of the data.

·         If you have several SQL Labels that query values from the same table the same way, then better use “Editable SQL Text Box“. As these are optimized for that case.

Sample:

“Select pipe_type from TT_PIPE where FID_PARENT = {FID}”

“Select pipe_color from TT_PIPE where FID_PARENT = {FID}”

“Select pipe_name from TT_PIPE where FID_PARENT = {FID}”

So only the result columns differ but the query “TT_PIPE where FID_PARENT = {FID}” is the same. If you use “Readonly SQL Textboxes” (SQL Labels) the system runs three Selects. If you use “Editable SQL Text Box“, then the system combines the three selects internally to one select:

“Select pipe_type,pipe_color,pipe_name from TT_PIPE where FID_PARENT = {FID}”

But this only works if the WHERE part “where FID_PARENT = {FID}” is exactly the same. It has to be case sensitive and the same amount of blanks etc. Set the Property “Is Locked” to ‘TRUE’ so it acts like a SQL Label.

·         Invalid select statments slow down your performance!

If you see “#ERROR#” in a SQL Label, then this means the select is invalid. This can happen to some of the records, maybe if the column you used in the select statement is NULL. To check the error, go to the record where you see the error and open the Form designer (CTRL+F). Then check the select in the SQL Assistant by executing it.

How to load point coordinates in Topobase ?

Create an excel sheet with the following information (attribute names depend of the feature class):

NAME_NUMBER	GEOM.X	GEOM.Y	Z	ORIENTATION
MyNewPoint	200000	600000	400	57

Load the data in the point Form through Excel Import.

And that’s it !

Oracle Database Analysis – Finding bottlenecks

When a database is up and running, every connected process is either busy performing work or waiting to do so. A process that is waiting may mean nothing in the overall scheme of things - the user may have left to get a cup of coffee, for example - or it can be an indicator that a database bottleneck exists. This is where wait-based or bottleneck analysis comes into play.

Waits are conditions where a session is waiting for something to happen. Waits can be caused by a number of things from slow disks, to locking situations to various kinds of internal Oracle contention.
The response time of a database operation consists of two factors, the service time (execution) plus the wait time:

Response Time = Service Time + Wait Time

How do you correctly practice bottleneck or wait-based analysis? First, it is imperative that you obtain not only the number of waits for a particular event but also the times involved for each wait. Having both lets you form a complete picture regarding the magnitude of wait-initiated performance degradation. (Of course, the CPU may be the actual bottleneck, but that's a subject for a different post.)

The view V$SYSTEM_WAIT_CLASS simply shows the wait times and counts across the new wait classes. You can use the following query to analyze the waits in your Oracle server:

select  WAIT_CLASS,

        TOTAL_WAITS,

        round(100 * (TOTAL_WAITS / SUM_WAITS),2) "WAITS %",

        TIME_WAITED,

        round(100 * (TIME_WAITED / SUM_TIME),2) "TIME %"

from

(select WAIT_CLASS,

        TOTAL_WAITS,

        TIME_WAITED

from    V$SYSTEM_WAIT_CLASS

where   WAIT_CLASS != 'Idle'),

(select  sum(TOTAL_WAITS) SUM_WAITS,

        sum(TIME_WAITED) SUM_TIME

from    V$SYSTEM_WAIT_CLASS

where   WAIT_CLASS != 'Idle')order by 5 desc;

A sample output could look like this:

WAIT_CLASS      TOTAL_WAITS WAITS % TIME_WAITED TIME %

--------------- ----------- ------- ----------- ------

System I/O      80567       5.6     56291       31.24

Commit          58050       4.04    50071       27.79

Concurrency     50017       3.48    36772       20.41

User I/O        26845       1.87    17695       9.82

Configuration   7004        .49     10742       5.96

Other           10628       .74     4217        2.34

Network         1204195     83.71   2956        1.64

Application     1269        .09     1418        .79

In this example it's easy to see that the bulk of the overall wait time is due to system or user I/O waits. This method is much quicker and more accurate than tallying individual wait events to get a global picture. The following list explains the different wait classes in more detail:

Administrative (DBA tasks affecting others): Buffer pool resize, Offline rollback segment, Index rebuilds
Application: Row level locks or explicit locks
Commit: Redo log write confirmation
Concurrency: Buffer busy waits, Library cache latch
Configuration: Sizing for log buffer, log files, SGA, Contention for ST enqueue
Network: More data to/from database link/client, Remote archive destination
User I/O: Database file reads, single writes, Direct path reads/writes, BFILE reads
System I/O: Redo log writes, Archiving redo logs, Control file writes
Idle: Indicates that session is inactive, waiting for more work

You find more information about this topic in the article from Robin Schumacher.

Oracle Database Analysis – Time spent

In the past couple of blog posts I showed you ways to tune your Oracle Database. Now I would like to share some other insights into Oracle. In this post I explain how to determine which tasks Oracle has spent most of its time on (since the last startup).

It’s important (and in my opinion also interesting) to know if your Oracle database uses most of its time to parse and execute your SQL statements OR to execute PL/SQL and Java code (stored procedures and triggers such as Topobase Feature Rules).

With the following SQL statement we can find the answer to that question:

select STAT_NAME,

       ROUND((VALUE / 1000000), 2) "Time (sec)"

from   V$SYS_TIME_MODEL

order by 2 DESC

I have executed this SQL statement against our Oracle Performance Test server, so the results represent up to 200 concurrent Topobase Client users:

STAT_NAME                                          Time (sec)

-------------------------------------------------- ----------

DB time                                            8577.61

DB CPU                                             5134.29

parse time elapsed                                 5133.76

hard parse elapsed time                            4842.81

sql execute elapsed time                           4362.92

PL/SQL compilation elapsed time                    1477.02

hard parse (sharing criteria) elapsed time         739.79

background elapsed time                            671.44

PL/SQL execution elapsed time                      471.47

sequence load elapsed time                         116.87

Java execution elapsed time                        103.66

...

Interesting values for a DBA or a Software Developer on Topobase are the following:

DB time:
Amount of elapsed time (in microseconds) spent performing Database user-level calls. This does not include the time spent on instance background processes such as PMON.
Compare this value against all of the other values contained in this table to determine where the bulk of the time is being spent.

parse time elapsed:
Amount of elapsed time spent parsing SQL statements. It includes both soft and hard parse time.

sql execute elapsed time:
Amount of elapsed time SQL statements are executing. Note that for select statements this also includes the amount of time spent performing fetches of query results.
If it is extremely high, look for poorly performing SQL statements.

PL/SQL execution elapsed time:
Amount of elapsed time spent running the PL/SQL interpreter.

Java execution elapsed time:
Amount of elapsed time spent running the Java VM.

Oracle Database Tuning – Database Files

When working with huge datasets the task of disk-load balancing has become very important. Such databases are often too big to be fully cached in the memory. But users still expect instant response times. This leads to a major problem--as we all know the most time consuming part of a transaction is the disk I/O. So the question is: how can we reduce the number of disk reads and writes?

The answer to that question is easy; if possible allocate more memory to Oracle (SGA size), see also the previous post about Oracle Database Buffer Cache Hit Ratio. Otherwise, try to reduce the time to read and write from your disks by distributing the database files among all available disks. In my opinion, the following rule of thumb optimizes performance:

• one schema per tablespace
• one tablespace per database file
• one database file per disk

One database file per disk reduces simultaneous I/O requests and improves performance. You can also reach this goal by disk striping, which spreads the load across multiple disks automatically.

Use the following SQL statement to gather the following information:

• How fast are the read and write accesses to the disks? You might reduce these values by maintaining your disks, for example, defragmenting.
• Does the read and write percentage of the database files (tablespace) seem to be OK? For example, if the SYSTEM database file has a lot of I/O operations, you may not have set the correct default and temporary tablespace for your Topobase schemas.
• Are your I/O operations equally distributed among your disks (if you have more than one disk)? If not, consider moving the most used (or all) database files to separate HDDs.

select NAME,

        PHYRDS "Physical Reads",

        round((PHYRDS / PD.PHYS_READS)*100,2) "Read %",

        round(READTIM/PHYRDS, 2) "Time per Read (sec/100)",

        PHYWRTS "Physical Writes",

        round(PHYWRTS * 100 / PD.PHYS_WRTS,2) "Write %",

        round(WRITETIM/PHYWRTS, 2) "Time per Write (sec/100)",

        fs.PHYBLKRD+FS.PHYBLKWRT "Total Block I/O's"

from   (select sum(PHYRDS) PHYS_READS,

               sum(PHYWRTS) PHYS_WRTS

        from    v$filestat

        ) pd,

        v$datafile df,

        v$filestat fs

where   df.FILE# = fs.FILE#

order   by fs.PHYBLKRD+fs.PHYBLKWRT desc