Download and convert to Geopackage
This downloads SWOT Pixel Cloud products from hydroweb.next (API-Key necessary) based on a region and a period of interest. Then is extracts information contained in the area of interest for your study, stores everything in a Geopackage Database for future use. Geopackage is a convenient data storage format, based on SQL, and is compatible with QGIS.
Setting the region and period of interest
Using a geopackage layer, preliminary created with, e.g. QGIS, to limit data download and database
[2]:
from pixcdust.downloaders.hydroweb_next import PixCDownloader
import geopandas as gpd
from datetime import datetime
/home/hysope2/SRC/preprocess_swot_pixc/pixcdust/.venv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
[12]:
# reading the area of interest polygon (could have been set)
gdf_geom = gpd.read_file('/home/hysope2/STUDIES/SWOT_Panama/DATA/aoi.gpkg')
# Limiting time period
dates = (
datetime(2023,4,6),
datetime(2023,4,15),
)
Download
This will unfortunately lead to downloading many big files (that will be removed later). This is the only way right now, but the hydroweb.next team is working on improving that.
[13]:
pixcdownloader = PixCDownloader(
gdf_geom,
dates,
verbose=0,
path_download='/tmp/pixc',
)
pixcdownloader.search_download()
Extraction
Now we have all necessary files, let us extract key variables within area of interest in a geopackage database. This geopackage format is quite efficient (though not the most efficient), and may easily be visualized in, e.g., QGIS We are using the same geodataframe to limit the data to the area of interest
[14]:
from pixcdust.converters.gpkg import PixCNc2GpkgConverter
from glob import glob
[15]:
pixc = PixCNc2GpkgConverter(
glob(pixcdownloader.path_download+'/*/*nc'),
"/tmp/my_awesome_pixc_gpkg.gpkg",
variables=['height', 'sig0', 'classification'],
area_of_interest=gdf_geom,
mode='o',
)
pixc.database_from_nc()
12%|█▏ | 2/17 [00:19<02:08, 8.56s/it]
--File /tmp/pixc/URN_FEATURE_DATA_swot_087c8ba8-8f8b-374e-8c6d-380199f82771_V1-URN_FEATURE_DATA_swot_087c8ba8-8f8b-374e-8c6d-380199f82771_V1/SWOT_L2_HR_PIXC_486_009_171L_20230410T032638_20230410T032647_PGC0_01.nc combined with area of interest returned empty. Skipping it
18%|█▊ | 3/17 [00:22<01:23, 5.98s/it]
--File /tmp/pixc/URN_FEATURE_DATA_swot_0f71e233-c0f2-3b60-a8e7-b79fd62bddee_V1-URN_FEATURE_DATA_swot_0f71e233-c0f2-3b60-a8e7-b79fd62bddee_V1/SWOT_L2_HR_PIXC_485_009_171L_20230409T033600_20230409T033609_PGC0_01.nc combined with area of interest returned empty. Skipping it
24%|██▎ | 4/17 [00:25<01:01, 4.74s/it]
--File /tmp/pixc/URN_FEATURE_DATA_swot_2f4a4153-ed7b-3bb3-ad4d-0976b643125d_V2-URN_FEATURE_DATA_swot_2f4a4153-ed7b-3bb3-ad4d-0976b643125d_V2/SWOT_L2_HR_PIXC_484_009_171L_20230408T034522_20230408T034531_PGC0_01.nc combined with area of interest returned empty. Skipping it
41%|████ | 7/17 [01:00<01:25, 8.54s/it]
--File /tmp/pixc/URN_FEATURE_DATA_swot_4f1cbd93-176a-364a-924b-d0e035996e63_V1-URN_FEATURE_DATA_swot_4f1cbd93-176a-364a-924b-d0e035996e63_V1/SWOT_L2_HR_PIXC_483_009_171L_20230407T035443_20230407T035453_PGC0_01.nc combined with area of interest returned empty. Skipping it
53%|█████▎ | 9/17 [01:19<01:07, 8.46s/it]
--File /tmp/pixc/URN_FEATURE_DATA_swot_624906c9-414e-3e3a-92f4-153de04c9d00_V1-URN_FEATURE_DATA_swot_624906c9-414e-3e3a-92f4-153de04c9d00_V1/SWOT_L2_HR_PIXC_490_009_171L_20230414T024910_20230414T024920_PGC0_01.nc combined with area of interest returned empty. Skipping it
59%|█████▉ | 10/17 [01:22<00:46, 6.69s/it]
--File /tmp/pixc/URN_FEATURE_DATA_swot_701bcc33-4432-3ff3-9908-7489aeceff0b_V1-URN_FEATURE_DATA_swot_701bcc33-4432-3ff3-9908-7489aeceff0b_V1/SWOT_L2_HR_PIXC_489_009_171L_20230413T025832_20230413T025842_PGC0_01.nc combined with area of interest returned empty. Skipping it
65%|██████▍ | 11/17 [01:24<00:32, 5.49s/it]
--File /tmp/pixc/URN_FEATURE_DATA_swot_7d92152a-1890-3555-9308-2a9e2bcfbb00_V1-URN_FEATURE_DATA_swot_7d92152a-1890-3555-9308-2a9e2bcfbb00_V1/SWOT_L2_HR_PIXC_488_009_171L_20230412T030754_20230412T030804_PGC0_01.nc combined with area of interest returned empty. Skipping it
82%|████████▏ | 14/17 [01:59<00:25, 8.46s/it]
--File /tmp/pixc/URN_FEATURE_DATA_swot_c614f219-595c-3afe-97d4-ed276a73489e_V1-URN_FEATURE_DATA_swot_c614f219-595c-3afe-97d4-ed276a73489e_V1/SWOT_L2_HR_PIXC_487_009_171L_20230411T031716_20230411T031725_PGC0_01.nc combined with area of interest returned empty. Skipping it
100%|██████████| 17/17 [02:47<00:00, 9.88s/it]
database has been succesfully created, we can remove the raw files
[16]:
# import shutil
# shutil.rmtree('/tmp/pixc')
Read the database
previous steps are not necessary
Now we can open this database in a GeoDataFrame, load it in, e.g., QGIS, etc.
[17]:
from pixcdust.readers.gpkg import PixCGpkgReader
# nb: you may specify
pixc_read = PixCGpkgReader(
"/tmp/my_awesome_pixc_gpkg.gpkg"
)
pixc_read.read()
pixc_read.data
0%| | 0/9 [00:00<?, ?it/s]100%|██████████| 9/9 [00:02<00:00, 4.25it/s]
[17]:
| height | sig0 | classification | geoid | latitude | longitude | wse | geometry | |
|---|---|---|---|---|---|---|---|---|
| 0 | 12.554601 | 40.937172 | 4.0 | 12.927674 | 8.901639 | -79.547015 | -0.373074 | POINT (-79.54701 8.90164) |
| 1 | 12.654260 | 31.059845 | 4.0 | 12.927361 | 8.901656 | -79.547131 | -0.273101 | POINT (-79.54713 8.90166) |
| 2 | 12.695892 | 16.790562 | 4.0 | 12.927069 | 8.901672 | -79.547239 | -0.231176 | POINT (-79.54724 8.90167) |
| 3 | 12.558355 | 13.450170 | 4.0 | 12.927868 | 8.901819 | -79.546884 | -0.369513 | POINT (-79.54688 8.90182) |
| 4 | 12.593613 | 36.509998 | 4.0 | 12.927578 | 8.901835 | -79.546991 | -0.333965 | POINT (-79.54699 8.90184) |
| ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 1827965 | 103.165474 | 2.514047 | 1.0 | 12.320300 | 9.160676 | -79.667186 | 90.845177 | POINT (-79.66719 9.16068) |
| 1827966 | 50.091274 | -2.230366 | 1.0 | 12.317310 | 9.160793 | -79.667979 | 37.773964 | POINT (-79.66798 9.16079) |
| 1827967 | 50.749222 | 0.185611 | 1.0 | 12.316775 | 9.160814 | -79.668121 | 38.432446 | POINT (-79.66812 9.16081) |
| 1827968 | 26.998960 | -2.889698 | 1.0 | 12.325519 | 9.160471 | -79.665802 | 14.673442 | POINT (-79.66580 9.16047) |
| 1827969 | 69.837334 | -7.154676 | 1.0 | 12.308424 | 9.161142 | -79.670342 | 57.528908 | POINT (-79.67034 9.16114) |
1827970 rows × 8 columns
Enjoy!