Python code snippet : Merge multiple CSVs (Column Wise) into a large CSV file using ID

Following is the python code snippet to merge multiple CSVs into a large CSV (column wise) using CSV's unique ID field. All input CSV's must have same length.

 import csv

 import itertools

 outCombinedStatTxtName ="FormattedTxtResults.csv"  
 tempCsvFiles = glob.glob(str(TempFolderPath)+'\*.csv')  
 openedFileList = [open(fn, 'rb') for fn in tempCsvFiles]  
 readers = [csv.reader(fn) for fn in openedFileList]  
   
 result = open(outCombinedStatTxtName, 'wb')  
 writer = csv.writer(result,delimiter=',')  
   
 multipleIndices = False  
 for row_chunks in itertools.izip(*readers):  
   tempFormattedRow = list(itertools.chain.from_iterable(row_chunks))  
   fidIndices = [i for i, x in enumerate(tempFormattedRow) if x == "ID"]  
   if(len(fidIndices) > 1):  
     fidIndices.pop(0)  
     redundantIndices = tuple(fidIndices)  
     multipleIndices = True  
       
   if(multipleIndices):  
     tempFormattedRow = [ tempFormattedRow[i] for i in xrange(len(tempFormattedRow)) if i not in set(redundantIndices) ]      
     writer.writerow(tempFormattedRow)  
   else:  
     writer.writerow(tempFormattedRow)  
 result.close()     
 [fn.close() for fn in openedFileList]  

Read More

Code Snippet: Select feature polygon(s) on Mouse Click ArcObjects C#

Code snippet of custom feature(s) selection tool on mouse click and highlight it using ArcObjects and C#.

 class SelectFeatureTool : ESRI.ArcGIS.Desktop.AddIns.Tool  
   {  
     protected override void OnMouseDown(MouseEventArgs arg)  
     {  
       IMxDocument mxDocument = ArcMap.Application.Document as IMxDocument;  
       IActiveView activeView = mxDocument.ActiveView;  
       IMap map = mxDocument.FocusMap;  
       ILayer layer = map.get_Layer(0); //Get 1st Layer  
       IFeatureLayer featureLayer = (IFeatureLayer) layer;  
   
       IPoint identifyPoint = activeView.ScreenDisplay.DisplayTransformation.ToMapPoint(arg.X, arg.Y);  
       ESRI.ArcGIS.Carto.IIdentify identifyLayer = (IIdentify)layer;  
       IArray array = identifyLayer.Identify(identifyPoint);  
         
       int oid = -1;  
       if (array != null)  
       {  
         object obj = array.get_Element(0);  
         IFeatureIdentifyObj fobj = obj as IFeatureIdentifyObj;  
         IRowIdentifyObject irow = fobj as IRowIdentifyObject;  
         IFeature feature = irow.Row as IFeature;  
         oid = feature.OID;  
       }  
       HighlightClickedFeature(featureLayer, oid, activeView);  
     }  
   

Read More

Catch Exception - Attempted to read or write protected memory in ESRI ArcObjects .Net 4.0 Framework

First - Don't write the code to throw "Attempted to read or write protected memory" ..it is bad.

Second - If you are working on legacy codes and COM objects that throws "Attempted to read or write protected memory" while writing ArcMap add-ins, catch exceptions to prevent application crash, in my case ArcMap.


Step #1 - Add following snippet to config file - app.config for Arc-addin

<configuration>
   <runtime>
      <legacyCorruptedStateExceptionsPolicy enabled="true" />
   </runtime>
</configuration>

Step #2

Add -

[HandleProcessCorruptedStateExceptions] 
[SecurityCritical]

on the top of function you are tying catch the exception

[HandleProcessCorruptedStateExceptions]
[SecurityCritical]
public IFeatureLayer GetOrCreateFeatureLayer(string path, esriGeometryType type, ISpatialReference sr)
{

//body

}

Read More

Good compatibility of latest vagrant and virtual box

Continuing from my older post  on Vagrant 1.7.2 / 1.9.2 and Virtualbox 4.3.12  / 5.1.14. I upgraded into Vagrant 1.9.2 and 5.0.x and 5.1.x , because of security vulnerabilities on those Virtualbox versions, but my shell provisions and puppet provision that used to work on old setting suddenly stopped working and tried various workaround to make them working but failed until Virtualbox 5.1.18. I use chocolaty, to upgrade/install vagrant and virtual box in Windows 7 host and able to fire 'vagrant up' to provision my new guest.

Step 1

Install chocolaty in host machine using cmd.exe (Run elevated)

@powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

  choco -v (Just to make sure chocolaty installed correctly)
  
  0.10.3 (as of 3/16/2017)

Step 2
Open Windows' power shell (Run elevated) 
1. Install or upgrade oracle virtual box

choco install virtualbox -version 5.1.18
                                                                                                         5.1.18 r114002
choco upgrade virtualbox

2. Install or upgrade vagrant

choco install vagrant -version 1.9.2

choco upgrade vagrant

3. Restart the machine

4. Vagrant up ....Boom all old scripts starting work again!

Read More

List of things appropriate and not appropriate for ArcGIS from Citrix Environment

Appropriate for Citrix

·         making maps with existing data

·         querying and selecting spatial data

·         creating GIS points from XY coordinates (GPS files)

·         Sub -setting data by selection and export

·         Creating or editing GIS data feature by feature

·         Buffering small data layers (few features, simple features)

·         Joins on attribute tables

Not Appropriate for Citrix

·         Geoprocessing (Intersect, Union, Identity …)

·         Sub -setting data by Clipping

·         Spatial joins

·         Spatial Analyst

·         Buffering large or complex data layers

 Source

Read More

Code snippet : Create/update polygon attribute fields using Ogr and ESRI Arcobjects

A code snippet example to check if a feature is standalone ESRI Shape or featureclass inside ESRI Gdb, then add new filed or update attribute value in attribute table .... standalone shape/features are created/updated using GDAL/OGR implementation and feature/featureclass stored in ESRI personal GDB are created/updated using ESRI ArcObjects.  Any solution to replace later with GDAL/Ogr is highly appreciated.  Please comment below if you find one.

 public void AddUpdateAttributeField(string oldFeatureFile)  
     {  
       DriverUtils.RegisterOgrDriver();  
       DataSource dataSource;  
       Layer layer;  
       var isShapeFile = IsShapeInGdb(oldFeatureFile);  
       if (isShapeFile)  
       {  
         dataSource = Ogr.Open(oldFeatureFile, 1); //second argument in open specifies mode of data, 1 RW & 0 readonly mode  
         layer = dataSource.GetLayerByIndex(0);  
         FieldDefn gdalFiedlDefn = new FieldDefn("FID_GDAL",FieldType.OFTInteger);  
         layer.CreateField(gdalFiedlDefn, 0);  
         Feature feature = layer.GetNextFeature();  
         while (feature!= null)  
         {  
           feature.SetField("FID_GDAL",feature.GetFID()); // Add FID shapefile  
           layer.SetFeature(feature);  
           feature = layer.GetNextFeature();  
         }  
         dataSource.FlushCache();  
       }  
       else  
       {  
         try  
         {  
           EnableEsriLiscences();  
           string gdbPath = Path.GetDirectoryName(oldFeatureFile);  
           string featureName = Path.GetFileNameWithoutExtension(oldFeatureFile);  
           IWorkspaceFactory workspaceFactory = new FileGDBWorkspaceFactory();  
           IFeatureWorkspace featureWorkspace = (IFeatureWorkspace)workspaceFactory.OpenFromFile(gdbPath, 1);  
           IFeatureClass featureClass = featureWorkspace.OpenFeatureClass(featureName);  
           IFields fields = featureClass.Fields;  
           if (fields.FindField("FID_GDAL") == -1)  
           {  
             // Create a Int field called "FID_GDAL" for the fields collection

Read More

Code snippet: Create new Field in a Shape File using GDAL/OGR in C#

Add new field in existing shape file using OGR in C#.

 public void AddAttributeField(string oldShapeFile)  
     {  
       Ogr.RegisterAll();  
       DataSource dataSource = Ogr.Open(oldShapeFile, 1); //second argument in open specifies mode of data, 1 RW & 0 readonly mode  
       Layer layer = dataSource.GetLayerByIndex(0);  
       FieldDefn gdalFiedlDefn = new FieldDefn("NEW_FIELD",FieldType.OFTInteger);  
       layer.CreateField(gdalFiedlDefn, 1);  
       Feature feature = layer.GetNextFeature();  
       while (feature!= null)  
       {  
         feature.SetField("NEW_FIELD",feature.GetFID()); // Populate new field with feature FID  
         layer.SetFeature(feature);  
         feature = layer.GetNextFeature();  
       }  
       dataSource.FlushCache();  
     }  

Read More

ESRI FeatureClass vs Feature Layer

Feature Layer: A layer that references a set of feature data. Feature data represents geographic entities as points, lines, and polygons.

Feature Class:In ArcGIS, a collection of geographic features with the same geometry type (such as point, line, or polygon), the same attributes, and the same spatial reference. Feature classes can be stored in geodatabases, shapefiles, coverages, or other data formats. Feature classes allow homogeneous features to be grouped into a single unit for data storage purposes. For example, highways, primary roads, and secondary roads can be grouped into a line feature class named "roads." In a geodatabase, feature classes can also store annotation and dimensions.

Rasterized Feature Layer: A feature layer in ArcGlobe that exists as points, lines and polygons but is rendered as cell data. When layers are added to ArcGlobe, they may automatically be rendered in raster format to retain their cartographic symbology.

Read More

Register Database to ArcGIS server

There are two methods to register Database to ArcGIS Server
A) Register via ArcGIS Desktop

• Double click to ensure ArcGIS Server Instance is connected to ArcGIS Desktop
• Right Click to Publisher connection to ArcGIS Server
• Then Click ArcGIS Server Properties to open ArcGIS Server Properies
• Add Data Store in Registered Databases

ESRI Video on Registering database data with ArcGIS 10.1 for Server

B) Register via ArcGIS Server Manager Web Interface

SOURCE: http://gisdeveloper.blogspot.com/2014_04_01_archive.html ( Copied here for my future reference)
REGISTER DATABASE TO ARCGIS SERVER

While publishing data (from database) to ArcGIS server it gives you a warning that

24011: Data source is not registered with the server and data will be copied to the server

How to resolve this ??

Well there are different approaches, here is the one which i followed. But before doing anything Please Note:
Note: The client side libraries need to be copied to ArcGIS server's bin folder. What does this mean ??
Well you have to copy the libraries (obtained from esri's customer portal Or simply google it "PostgreSQLClientLibs922" Or PostgreSQL libraries for ArcGIS server)
Copy the correct version of libraries to
C:\Program Files\ArcGIS\Server\bin   

Note to see the correct version of libraries click here.

Now,

• Open ArcGIS server Manager from (prefer Internet Explorer)http://localhost:6080/arcgis/manager/site.html
• Sign in to your ArcGIS server account
• On home Page under Directories tab click on Data Store
• Click on Registered Database

Read More

Postgresql tips

Start to postgresql :  sudo -u postgres psql postgres

List all databases : 

• \list or \l: list all databases
• \dt: list all tables in the current database

Type \q and then press ENTER to quit psql
Connect to database : \connect database_name
Drop all tables from a database:  drop schema public cascade;

Read More