Data Science Basic Crop Analysis with UAV Imagery and Micasense Altum

JP Uncategorized

https://github.com/dronemapper-io/CropAnalysis

A jupyter notebook with crop analysis algorithms utilizing digital elevation models, dtm and multi-spectral imagery (R-G-B-NIR-Rededge-Thermal) from a MicaSense Altum sensor processed with DroneMapper Remote Expert.

Due to limitations on git file sizes, you will need to download the GeoTIFF data for this project from the following url: https://dronemapper.com/software/DroneMapper_CropAnalysis_Data.zip. Once that has been completed, extract the TIF files into the notebook data directory matching the structure below.

Included Data

  • data/DrnMppr-DEM-AOI.tif – 32bit georeferenced digital elevation model
  • data/DrnMppr-ORT-AOI.tif – 16bit georeferenced orthomosaic (Red-Green-Blue-NIR-Rededge-Thermal)
  • data/DrnMppr-DTM-AOI.tif – 32bit georeferenced dtm
  • data/plant_count.shp – plant count AOI
  • data/plots_1.shp – plot 1 AOI
  • data/plots_2.shp – plot 2 AOI
  • output/*.csv – output csv of dataframes

Algorithms

  • plot volume/biomass
  • plot canopy height
  • plot ndvi zonal statistics
  • plot thermals
  • plant count

Notes

These basic algorithms are intended to get you started and interested in multi-spectral processing and analysis.

The orthomosaic, digital elevation model, and dtm were clipped to an AOI using GlobalMapper. The shapefile plots were also generated using GlobalMapper grid tool. We highly recommend GlobalMapper for GIS work!

We cloned the MicaSense imageprocessing repository and created the Batch Processing DroneMapper.ipynb notebook which allows you to quickly align and stack a Altum or RedEdge dataset creating the correct TIF files with EXIF/GPS metadata preserved. These stacked TIF files are then directly loaded into DroneMapper Remote Expert for processing.

This notebook assumes the user has basic knowledge of setting up their python environment, importing libraries and working inside jupyter.

Do More!

Implement additional algorithms like NDRE or alternative methods for plant counts. Submit a pull request to the repo!

Load Digital Elevation Model and Orthomosaic

Orthomosaic-DigitalElevationModel

Load Plot 1 AOI and Generate NDVI

ndvi_zonal_statistics

Generate NDVI Zonal Statistics For Each Plot

 ndvi_zonal_statistics_plots
geometryLAYERMAP_NAMENAMEminmaxmeancountstdmedian
0POLYGON Z ((289583.708 5130289.226 0.000, 2895…Coverage/QuadUser Created Features20.1884610.8730920.43855944440.0789210.436500
1POLYGON Z ((289588.705 5130289.052 0.000, 2895…Coverage/QuadUser Created Features30.1932140.8879710.44528244400.0910900.425304
2POLYGON Z ((289593.702 5130288.877 0.000, 2895…Coverage/QuadUser Created Features40.2322220.8901470.55286444400.1124400.519746
3POLYGON Z ((289598.699 5130288.703 0.000, 2896…Coverage/QuadUser Created Features50.0908250.8650830.53029544440.1105700.515392
4POLYGON Z ((289603.696 5130288.528 0.000, 2896…Coverage/QuadUser Created Features60.1046970.9224500.53666044420.1327310.495813

Load Plot 2 AOI & Compute DEM Canopy Mean Height For Each Plot

 canopy_height_dem
geometryLAYERMAP_NAMENAMEminmaxmeancountstdmedian
0POLYGON Z ((289707.875 5130279.812 1182.502, 2…Coverage/QuadUser Created Features – Coverage/Quad1360.129120363.381683361.14129433181.091593360.441467
1POLYGON Z ((289712.189 5130279.586 1190.569, 2…Coverage/QuadUser Created Features – Coverage/Quad2360.131866363.382446361.71029733161.215926361.927017
2POLYGON Z ((289716.503 5130279.360 1183.212, 2…Coverage/QuadUser Created Features – Coverage/Quad3360.117279363.384766361.13859233101.122890360.425781
3POLYGON Z ((289720.817 5130279.134 1182.668, 2…Coverage/QuadUser Created Features – Coverage/Quad4360.110443363.387207361.91543633221.258644362.585251
4POLYGON Z ((289725.131 5130278.908 1182.782, 2…Coverage/QuadUser Created Features – Coverage/Quad5360.006683363.377991361.55850133201.305164360.546524

Compute Thermal Mean For Each Plot

The thermal band (5) in the processed orthomosaic shows stitching artifacts which could likely be improved using more accurate pre-processing alignment and de-distortion algorithms. You can find more information about these functions in the MicaSense imageprocessing github repository. See notes at the top of this notebook.

 
geometryLAYERMAP_NAMENAMEminmaxmeancountstdmedian
0POLYGON Z ((289707.875 5130279.812 1182.502, 2…Coverage/QuadUser Created Features – Coverage/Quad130008.030431.030196.6286923318120.98205830193.0
1POLYGON Z ((289712.189 5130279.586 1190.569, 2…Coverage/QuadUser Created Features – Coverage/Quad230068.030560.030333.1927023316123.85609330332.0
2POLYGON Z ((289716.503 5130279.360 1183.212, 2…Coverage/QuadUser Created Features – Coverage/Quad329792.030645.030266.0302113310170.83120730275.5
3POLYGON Z ((289720.817 5130279.134 1182.668, 2…Coverage/QuadUser Created Features – Coverage/Quad429790.030700.030386.1372673322201.26691930391.0
4POLYGON Z ((289725.131 5130278.908 1182.782, 2…Coverage/QuadUser Created Features – Coverage/Quad529618.030691.030209.9045183320292.29939230262.0

Load Plot 1 AOI & Compute Volume/Biomass For Each Plot

biomass
geometryLAYERMAP_NAMENAMEminmaxmeancountsumvolume_m3area_m2
0POLYGON Z ((289583.708 5130289.226 0.000, 2895…Coverage/QuadUser Created Features2-0.1534733.5255431.53252444446810.53848376.65036850.0
1POLYGON Z ((289588.705 5130289.052 0.000, 2895…Coverage/QuadUser Created Features3-0.0944823.5752261.64666444407311.18869082.28502150.0
2POLYGON Z ((289593.702 5130288.877 0.000, 2895…Coverage/QuadUser Created Features4-0.0706483.9924931.88459644408367.60827694.17467650.0
3POLYGON Z ((289598.699 5130288.703 0.000, 2896…Coverage/QuadUser Created Features50.0329284.5759892.969443444413196.202637148.51891550.0
4POLYGON Z ((289603.696 5130288.528 0.000, 2896…Coverage/QuadUser Created Features60.0741885.1054083.155879444214018.412506157.77261750.0

Load Plant Count AOI & Count Plants

plant_count
plant_count
Plant count: 310
UTMXUTMY
0289622.4111715130240.899705
1289621.8497055130244.325966
2289622.1015695130248.858655
3289621.8650865130253.863387
4289621.4362805130258.158493

Thanks! Keep an eye out for future notebooks and algorithms.

Using the Max Flow/Min Cut DEM algorithm for accurate volumetrics

JP Uncategorized

capacity-map

Figure 1

It's that time of the season again where water resource managers are checking their emptied reservoirs for possible silt-in and verification of water holding capacities. We recently flew one of these reservoirs on the south side of the Grand Mesa in western Colorado with our Phantom 3 and a set of ground surveyed aerial targets. The imagery and control were processed using two different DEM generation algorithms, the results of which are presented and discussed here.

Figure 1 illustrates the ortho of the area with the ground surveyed targets and other points depicted as black dots. Figure 2 shows the DEM construction using our standard algorithm. The black oval highlights DEM decorrelation where the vegetation and shadows obscure the NW bank of the reservoir causing erroneous elevation rendering. Figure 3 shows the DEM construction utilizing the max flow/min cut DEM algorithm with greatly enhanced correlation in the same area previously highlighted.

We use Global Mapper to generate contours at specific elevations of interest ranging from reservoir drain or dead pool to maximum capacity or spill elevation. Each contour generated is converted to a flat water elevation surface and the volume between that surface and the DEM reservoir surface is computed. Take a look at what happens when contours are generated on a locally decorrelated DEM in Figure 4. One can see the contours are moving into the NW vegetated area which produces higher volumetric and surface area estimates than reality. As a comparison Figure 5 using the Max/Flow/Min Cut algorithm demonstrates more accurate rendering of the contours and subsequently higher accuracy volume estimates.

Image

Figure 2

Image

Figure 3

Image

Figure 4

Image

Figure 5

Drone Mapping Video Tutorials

JP Uncategorized

We are generating a set of video tutorials to help users get setup and run your imagery collections efficiently using REMOTE EXPERT or RAPID. We hope that these will provide some insight on how your various processing needs can be met and the options you can select for your specific scene. We understand we're not all photogrammetry experts and provide these tutorials to help you select the best options for the dataset you collected and possibly more importantly, how you can modify your future collections to produce acceptable products for your applications and customers.

Please click on this link to access videos by topic: https://dronemapper.com/video-tutorials.

Check back often as we will updating this link with more material. Feel free to contact us with comments, critique and other topics you want to see.covered.

The very best from us, The DroneMapper Team

RAPID Update

JP Uncategorized

DroneMapper has just released an updated version of RAPID with full photogrammetry capability addressing affordability for smaller areas of interest. The RAPID Windows application provides functionality equivalent to REMOTE EXPERT:

  • Selectable DEM and Ortho output resolutions,
  • 64-bit Point Cloud outputs - traditional and textured meshes,
  • Ground control pre-processing for high geo-spatial accuracy,
  • Processing of RTK and PPK image geotags,
  • Processing of nadir as well as oblique collections,
  • Multi-spectral image processing,
  • Ortho seamline feathering and blending,
  • Frequent software enhancements at no additional cost.
RAPID is licensed for 1 year at a cost of $159. The application will ingest up to 250 images per project and run on modest computing resources.

When you download the software and install try running an imagery set of your own or use one of our example sets to produce a preview. RAPID will ONLY produce a preview and requires license purchase to complete the DEM and Ortho. When the DEM "go" button is clicked a Paypal window will open prompting payment. Once paid an activation code will be sent via email for your yearly license. DroneMapper will only accept Paypal payments for RAPID.

The DroneMapper Team!

NodeMICMAC – a new WebODM Node

JP Uncategorized

Great post from Stephen and Piero over at the OpenDroneMap project! We at DroneMapper are working closely with ODM and the rest of the open source community to empower you to build your own SaaS service, processing pipeline or simply contribute back to various development projects! We've built our commercial software chain around MicMac as others like PIX4D and Agisoft have. It is now time to start contributing back these improvements and continue building great products. We'll do our best to support ODM and NodeMICMAC while enhancing functionality, continuing integration and adding features.

Original Article @ OpenDroneMap

Origins

OpenDroneMap has an origins story rooted in a joke:

While the last decade has been dominated by the growing hegemony of the global base map, mapping will swing now for a while towards the principle of mapping the world, one organic pixel at a time. 2014 is the beginning of artisanal satellite mapping, where we discover the value in 1-inch pixels from personally and professionally flown unmanned aerial systems (drones). There is, as all things military-industrial, the dark side of drones. But as with all of these technologies, we will be discovering the great democratizing power of the artisanal, as applied to ‘satellite’ views.

OpenDroneMap anyone?

So many FOSS options… .

There are plenty of communities, spaces, and markets for Free and Open Source (FOSS) projects: in the web map rendering world we have MapServer and a dozen upstarts. In the map javascript world OpenLayers, Leaflet, and the GL ilk and more. At the lower level, we have JTS Topology Suite, GEOS, and all their derivatives.

After making the 2014 prediction post and then deciding to start OpenDroneMap, I had a doubt: what if someone comes along and creates something better? What if something better already exists? What if there is no point to the work? And then I remembered all the above and relaxed. Also, if someone comes along and creates something better, in the informal parlance: yay for the world!

The challenge

The reality when I started the project was there was an existing project that was FOSS and was photogrammetry for drones and other small cameras: MICMAC. It’s exquisite: great quality, fully FOSS being a CeCILL-B (I like to think of it as a French version of the GPL (edit — it’s the French version of the lesser GPL), but IANAFL [I am not a French Lawyer] and not completely sure that’s correct), and at the time, difficult to use for non-French speakers.

MICMAC has evolved a lot since then, with better docs and community posts in both French and English, however usage of it can still be a challenge. Free and Open Source is hard. It can be hard for users, it can be hard for maintainers. So, it is such a relief when FOSS becomes ever so much easier.

The hook

So, it is with some excitement that I turn your attention to NodeMICMAC. NodeMICMAC is a fork of NodeODM, and thus provides web API access to MICMAC, in the same way that NodeODM does for OpenDroneMap’s command line ODM application.

NodeMICMAC makes using MICMAC really easy, and should slot into the OpenDroneMap ecosystem pretty seamlessly, thanks to JP and the folks at DroneMapper.

When we spoke with JP, I was curious about his motives: this is such a cool move that changes the industry. Why? The answer: for the same reasons that we work on OpenDroneMap, to grow this really cool open photogrammetric ecosystem.

(Sidenote: check out DroneMapper’s geoBits: ArUco Ground Control Point (GCP) Targets and Detection for Aerial Imagery — more on that later — so cool!).

Results

But, you may ask, how does it stack up? Frighteningly well. If you have been paying attention to the improvements in OpenDroneMap the last couple of years, you may have noted orders of magnitude improvements in every step in processing. Even with these, MICMAC shows us how a mature photogrammetry project should display its wares.

Point cloud comparisons ODM vs. MICMAC’s point cloud over a house:

Animation comparing point cloud for house from MICMAC to OpenDroneMap

ODM vs. MICMAC’s point cloud over a drainage:

Animation comparing point cloud for ditch from MICMAC to OpenDroneMap

Orthophoto Comparison

Orthophoto over truck, fence, road, vegetation:

Comparison of MICMAC and ODM orthophotos over road, fence, and truck

Orthophoto over a duplex house:

Comparison of orthophoto of house between MICMAC and OpenDroneMap

Takeaways

MICMAC is, as it’s reputation indicates, a bit of a grail. It gives us some very nice results, and now simply at the expense of spinning up a docker instance. Watch this space the next few weeks — Masserano Labs will be working with DroneMapper on integrating it into WebODM and quickly graduating it to a first class citizen of the OpenDroneMap ecosystem.

So, will NodeMICMAC replace NodeODM and all the work in ODM? Not so fast! There’s still space for what we’ve built. Remember my intro above? Of course you do. With upcoming capacity to handle massive datasets, NodeODM, PyODM, ODM, and other projects will still get our love. But as they say, if you can’t beat them, have them join you. Right? I think that’s the phrase… . Perhaps MICMAC isn’t joining OpenDroneMap, but we will be happy to fork their code and contribute back where we can, and thanks to JP and DroneMapper for making this possible!

Original Article @ OpenDroneMap by Stephen Mather