Indian Forum for Water Adroit

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Pankaj Dey

Pages: 1 2 [3] 4 5 ... 15
In this paper, we review the use of satellite-based remote sensing in combination with in situ data to inform Earth surface modelling. This involves verification and optimization methods that can handle both random and systematic errors and result in effective model improvement for both surface monitoring and prediction applications. The reasons for diverse remote sensing data and products include (i) their complementary areal and temporal coverage, (ii) their diverse and covariant information content, and (iii) their ability to complement in situ observations, which are often sparse and only locally representative. To improve our understanding of the complex behavior of the Earth system at the surface and sub-surface, we need large volumes of data from high-resolution modelling and remote sensing, since the Earth surface exhibits a high degree of heterogeneity and discontinuities in space and time. The spatial and temporal variability of the biosphere, hydrosphere, cryosphere and anthroposphere calls for an increased use of Earth observation (EO) data attaining volumes previously considered prohibitive. We review data availability and discuss recent examples where satellite remote sensing is used to infer observable surface quantities directly or indirectly, with particular emphasis on key parameters necessary for weather and climate prediction. Coordinated high-resolution remote-sensing and modelling/assimilation capabilities for the Earth surface are required to support an international application-focused effort.


Hydrological sciences / Acquire Landsat 8 Data: R Package
« on: December 10, 2018, 08:23:19 AM »
rLandsat makes it easy to search for Landsat8 product IDs, place an order on USGS-ESPA and download the data along with the meta information in the perfect format from R. Internally, it uses a combination of sat-api, espa-api and AWS S3 Landsat 8 metadata.

link to GitHub :

 Abstract. Earth system models (ESMs) are key tools for providing climate projections under different scenarios of human-induced forcing. ESMs include a large number of additional processes and feedbacks such as biogeochemical cycles that traditional physical climate models do not consider. Yet, some processes such as cloud dynamics and ecosystem functional response still have fairly high uncertainties. In this article, we present an overview of climate feedbacks for Earth system components currently included in state-of-the-art ESMs and discuss the challenges to evaluate and quantify them. Uncertainties in feedback quantification arise from the interdependencies of biogeochemical matter fluxes and physical properties, the spatial and temporal heterogeneity of processes, and the lack of long-term continuous observational data to constrain them. We present an outlook for promising approaches that can help quantifying and constraining the large number of feedbacks in ESMs in the future. The target group for this article includes generalists with a background in natural sciences and an interest in climate change as well as experts working in interdisciplinary climate research (researchers, lecturers, and students). This study updates and significantly expands upon the last comprehensive overview of climate feedbacks in ESMs, which was produced 15 years ago (NRC, 2003).
Citation: Heinze, C., Eyring, V., Friedlingstein, P., Jones, C., Balkanski, Y., Collins, W., Fichefet, T., Gao, S., Hall, A., Ivanova, D., Knorr, W., Knutti, R., Löw, A., Ponater, M., Schultz, M. G., Schulz, M., Siebesma, P., Teixeira, J., Tselioudis, G., and Vancoppenolle, M.: Climate feedbacks in the Earth system and prospects for their evaluation, Earth Syst. Dynam. Discuss.,

Link to the paper:

1. More than 1.9 million tiles of MODIS daily reflectance data were used to create a 16- year record (2001 -2016) of daily water map for the entire globe.

2. The global daily water data set can be openly downloaded by potential users.

3. Global inland water has a dramatic seasonal variation ranging from approximately 3.8 million km2 in September to 1.5 million km2 in February within an annual cycle.
4. Short duration water bodies, sea level rise effects, various types of rice field use can be detected from the daily water data set.
Link to paper:
Link to data:

The zone of deep air convection and heavy precipitation in the Earth’s tropics, known as the intertropical convergence zone (ITCZ), is characterized by seasonal southward and northward movement, following Sun’s radiation, as well as long-term meridional shifts, which greatly affect water availability in many regions around the world. This study proposes and applies a new probabilistic framework to track the ITCZ by jointly considering multiple physical variables to define its location. It also allows for detailed analysis of the intra-annual dynamics in all longitudes of the globe, while being computationally efficient and flexible in its implementation. We reveal a statistically significant southward trend in the location of the ITCZ over the central Pacific.

Link to paper:

Interesting information / M_Map: A Mapping Package for MATLAB
« on: December 02, 2018, 10:29:27 PM »
M_Map is a set of mapping tools written for Matlab (it also works under Octave). M_Map includes:
  • Routines to project data in 19 different projections (and determine inverse mappings), using spherical and ellipsoidal earth-models.
  • A grid generation routine to make nice axes with limits either in lat/long terms or in planar X/Y terms.
  • A coastline database (with 1/4 degree resolution).
  • A global elevation database (1 degree resolution).
  • Hooks into freely available high-resolution coastline and bathymetry databases.
  • Other useful stuff.

Hydrological sciences / Energy, rain, and the future
« on: December 01, 2018, 06:26:00 PM »
The intensification of precipitation over land owing to rising atmospheric temperature is expected to increase in the 21st century, but there are still many details about why and how much that need to be better understood. Richardson et al. used a suite of climate models to analyze the atmospheric energy budget and its effects on the fast (due to atmospheric components like carbon dioxide and sulfate) and slow (due to surface temperature changes) drivers of precipitation change. They discuss likely future changes over land and sea and the causes of the past and projected trends and find that the increase may become clearly observable by around 2050.
Link to paper:

We’ve all heard it before: “Yeah, but the climate has ALWAYS changed.”
Oh, really? Well, this timeline of Earth’s average temperature shows just how much we’ve influenced the climate. This epic webcomic was created by Randall Munroe, the artist behind xkcd, one of our favorite places for simplifying complicated scientific concepts.
It’s pretty long, but bear with us.

Hydrological sciences / hddtools: Hydrological Data Discovery Tools in R
« on: November 25, 2018, 11:13:58 AM »
hddtools stands for Hydrological Data Discovery Tools. This R package is an open source project designed to facilitate access to a variety of online open data sources relevant for hydrologists and, in general, environmental scientists and practitioners.
This typically implies the download of a metadata catalogue, selection of information needed, a formal request for dataset(s), de-compression, conversion, manual filtering and parsing. All those operations are made more efficient by re-usable functions.
Depending on the data license, functions can provide offline and/or online modes. When redistribution is allowed, for instance, a copy of the dataset is cached within the package and updated twice a year. This is the fastest option and also allows offline use of package's functions. When re-distribution is not allowed, only online mode is provided.
Link to manual:

High-resolution soil moisture/temperature (SM/ST) are critical components of the growing demand for fine-scale products over the Indian monsoon region (IMR) which has diverse land-surface characteristics. This demand is fueled by findings that improved representation of land-state help improve rainfall/flood prediction. Here we report on the development of a high-resolution (4 km and 3 hourly) SM/ST product for 2001–2014 during Indian monsoon seasons (June–September). First, the quality of atmospheric fields from five reanalysis sources was examined to identify realistic forcing to a land data assimilation system (LDAS). The evaluation of developed SM/ST against observations highlighted the importance of quality forcing fields. There is a significant relation between the forcing error and the errors in the SM/ST. A combination of forcing fields was used to develop 14-years of SM/ST data. This dataset captured inter-annual, intra-seasonal, and diurnal variations under different monsoon conditions. When the mesoscale model was initialized using the SM/ST data, improved simulations of heavy rain events was evident, demonstrating the value of the data over IMR.

Link to paper:

Link to data:

We conduct critical reviews on the experimental and theoretical methodologies on soil water density to identify their limitations, flaws, and uncertainties. We synthesize some recent findings on intermolecular forces, interfacial interactions, and soil water retention mechanisms to clarify molecular-scale physicochemical mechanisms governing the soil water density. We propose a unified model to quantify soil water density variation.


Observing surface water is essential for ecological and hydrological studies. This paper reviews the current status of detecting, extracting, and monitoring surface water using optical remote sensing, especially progress in the last decade. It also discusses the current status and challenges in this field. For example, it was found that pixel unmixing and reconstruction, and spatio-temporal fusion are two common and low-cost approaches to enhance surface water monitoring. Remote sensing data have been integrated with in situ river flow to model spatio-temporal dynamics of surface water. Recent studies have also proved that the river discharge can be estimated using only optical remote sensing imagery. This will be a breakthrough for hydrological studies in ungauged areas. Optical sensors are also easily obscured by clouds and vegetation. This limitation can be reduced by integrating optical data with synthetic aperture radar data and digital elevation model data. There is increasing demand of monitoring global water dynamics at high resolutions. It is now easy to achieve with the development of big data and cloud computation techniques. Enhanced global or regional water monitoring in the future requires integrated use of multiple sources of remote sensing data.


Deep learning (DL), a new generation of artificial neural network research, has transformed industries, daily lives, and various scientific disciplines in recent years. DL represents significant progress in the ability of neural networks to automatically engineer problem-relevant features and capture highly complex data distributions. I argue that DL can help address several major new and old challenges facing research in water sciences such as interdisciplinarity, data discoverability, hydrologic scaling, equifinality, and needs for parameter regionalization. This review paper is intended to provide water resources scientists and hydrologists in particular with a simple technical overview, transdisciplinary progress update, and a source of inspiration about the relevance of DL to water. The review reveals that various physical and geoscientific disciplines have utilized DL to address data challenges, improve efficiency, and gain scientific insights. DL is especially suited for information extraction from image-like data and sequential data. Techniques and experiences presented in other disciplines are of high relevance to water research. Meanwhile, less noticed is that DL may also serve as a scientific exploratory tool. A new area termed AI neuroscience, where scientists interpret the decision process of deep networks and derive insights, has been born. This budding subdiscipline has demonstrated methods including correlation-based analysis, inversion of network-extracted features, reduced-order approximations by interpretable models, and attribution of network decisions to inputs. Moreover, DL can also use data to condition neurons that mimic problem-specific fundamental organizing units, thus revealing emergent behaviors of these units. Vast opportunities exist for DL to propel advances in water sciences.


The Internet has provided IS researchers with the opportunity to conduct studies with extremely large samples, frequently well over 10,000 observations. There are many advantages to large samples, but researchers using statistical inference must be aware of the p-value problem associated with them. In very large samples, p-values go quickly to zero, and solely relying on p-values can lead the researcher to claim support for results of no practical significance. In a survey of large sample IS research, we found that a significant number of papers rely on a low p-value and the sign of a regression coefficient alone to support their hypotheses. This research commentary recommends a series of actions the researcher can take to mitigate the p-value problem in large samples and illustrates them with an example of over 300,000 camera sales on eBay. We believe that addressing the p-value problem will increase the credibility of large sample IS research as well as provide more insights for readers.

Link to the paper:

Pages: 1 2 [3] 4 5 ... 15