Water Adroit Forum

Materials and Methods => Models => Topic started by: ASHWATHI V K on April 07, 2017, 02:39:35 PM

Title: Statistical downscaling of GCMs
Post by: ASHWATHI V K on April 07, 2017, 02:39:35 PM
Hai,

I want to find impact of climate change on water balance components in a small basin. I am using SWAT hydrological model. To find impact of climate change, i have selected 5 GCMs (CNRM-CM5, MRI- CGCM3, MIROC- ESM, CESM-CAM5, NorESM- M) of CMIP5 project based on literature. Scenarios selected are RCP 2.6,4.5 and 8.5. I have following doubts regarding climate model:
1. Can the GCMs of CMIP5 project be statistically downscaled using SDSM downscaling model? (In example which in the model webpage, they have used only HADCM3)
2. Metereological parameter of SWAT are precipitation, min and max temperature, wind speed, humidiy and solar radiation. Is it necessary to downscale all parameters from GCMs to use it in SWAT? Or can we use only precipitation and temperature?
3. When downloading GCM output is it necessary to download files of all variables? Or need to download only precipitation and min and max temperature?
4. Which is the best and simple method to statistically downscale GCM output?
5. If we are using SDSM downscaling model, on what basis we have to select predictor variables?

Being new to climate models, I have only little idea on climate models. What is the best method to find impact of climate change on hydrology with hydro logical model selected being SWAT?
 
Please help me in clearing my doubts.

With Regards
Ashwathi


 
Title: Re: Statistical downscaling of GCMs
Post by: Alok Pandey on April 07, 2017, 03:18:40 PM
My view :

1. Can the GCMs of CMIP5 project be statistically downscaled using SDSM downscaling model? (In example which in the model webpage, they have used only HADCM3)

-> Yes.

2. Metereological parameter of SWAT are precipitation, min and max temperature, wind speed, humidiy and solar radiation. Is it necessary to downscale all parameters from GCMs to use it in SWAT? Or can we use only precipitation and temperature?

->  Since all these parameters are also provided by the GCM, it is advisable or preferred (not necessarily imposed) to use  them in analysis. However if any other model provides these parameters as long term forecasts, it can be used to enhance our perspectives on uncertainties associated with the models.

3. When downloading GCM output is it necessary to download files of all variables? Or need to download only precipitation and min and max temperature?

->  Better download only those files which will be used for analysis.

4. Which is the best and simple method to statistically downscale GCM output?

->  Every method has its pros and cons. "Best" word is very subjective here. Complexity of downscaling methods can indeed be discussed. Useful Link : http://wateradroit.com/forum/index.php/topic,706.msg2064.html#msg2064

5.  If we are using SDSM downscaling model, on what basis we have to select predictor variables?

->  Predictor variables are selected based on their strong relationships (linear and/or non-linear) with predictand. High Correlation (also supported by physics of nature) is one widely accepted measure.

P.S. Researchers are encouraged to add and/or correct points mentioned in this post.
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on April 11, 2017, 03:08:25 PM
Hai,

Thank you Alok for your reply.

Based on my understanding from literature, in statistical downscaling, if we want future precipitation, first we have to select predictor variables (pressure, humidity,air temp etc) from NCEP-NCAR reanalysis data and precipitation as predictand (station data) for same period. Then find the relation between predictand and predictor variable in NCEP-NCAR data. Then calibrate the model and then select same predictor variables from GCM (pressure, humidity,air temp etc) and find predictand (future station data). Is this true? Or is there any change from this?

Or should we relate predictor variable in NCEP- NCAR data and predictor variables in GCM?
Title: Re: Statistical downscaling of GCMs
Post by: Alok Pandey on April 12, 2017, 04:17:09 PM
Hello,

"Based on my understanding from literature, in statistical downscaling, if we want future precipitation, first we have to select predictor variables (pressure, humidity,air temp etc) from NCEP-NCAR reanalysis data and precipitation as predictand (station data) for same period. Then find the relation between predictand and predictor variable in NCEP-NCAR data. Then calibrate the model and then select same predictor variables from GCM (pressure, humidity,air temp etc) and find predictand (future station data). Is this true? Or is there any change from this?"
-> The above mentioned approach is widely used. Also many agencies now provide reanalysis data (i.e. NCEP-NCAR, ERA, JRA). The main assumptions behind this approach is that the relationship between reanalysis data (predictor) and station data (predictand) will be preserved (i.e. stationary relationship) in future and thus can be used as it is on GCM data to obtain final future projections of predictand.

"Or should we relate predictor variable in NCEP- NCAR data and predictor variables in GCM?"
-> I did not understand the rationale behind this? Where will we use this relationship in the analysis?
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on April 13, 2017, 11:41:56 AM
Hai,

Actually i want to find impact of climate change on water balance components in SWAT hydrological model. To find climate change, i have selected some GCMs. To use GCM i need to statistically downscale it using SDSM downscaling model. To statistically downscale GCM,  i have to select NCEP-NCAR reanalysis data. Is this procedure right?

There are a lot of files with different variable in NCEP-NCAR reanalysis data archive? Should i download entire files? How will we obtain NCEP -NCAR reanalysis data for the grid containing my study area. Being a fresher in Climate modelling, i am bit confused. Is there any link to obtain NCEP -NCAR reanalysis data.
Please help me.

With Regards
Ashwathi
Title: Re: Statistical downscaling of GCMs
Post by: Alok Pandey on April 14, 2017, 09:01:26 AM
Hello,

"There are a lot of files with different variable in NCEP-NCAR reanalysis data archive? Should i download entire files? How will we obtain NCEP -NCAR reanalysis data for the grid containing my study area."
-> No need to download entire files. NCEP-NCAR provides freedom to download specific variable (pressure, humidity,air temp etc.) of particular time scales (daily, monthly etc.) corresponding to region of interest. more information on below links :
1. http://wateradroit.com/forum/index.php/topic,47.msg98.html#msg98
2. http://wateradroit.com/forum/index.php/topic,485.msg1584.html#msg1584
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on April 27, 2017, 12:23:37 PM
Hai,

If there is only one grid of reanalysis data in the study area, is it necessary to do principal component analysis before doing regression?


With regards
Ashwathi
Title: Re: Statistical downscaling of GCMs
Post by: Alok Pandey on April 27, 2017, 03:34:25 PM
Hello,

In downscaling, we usually take grids which encompass the whole study area and not just the grid which falls into it. This way the minimum number of grids for the study comes out to be 4 (for one climate variable). Thus more number of variables necessitates use of PCA.
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on May 03, 2017, 01:45:16 PM
Hai,

I have NCEP/NCAR reanalysis data in netcdf format. I want to convert it into csv file. I have a script in R programming to convert netcdf to csv. But since the data in netcdf file is too large which exceeds number columns in csv file, i am not able get entire data in csv format. How can i split the data in nc file to multiple sheets of csv file.

Script is as follows:

library(ncdf4)
getwd()
workdir <- "I:\\NCEP\\"
setwd(workdir)
ncin <- nc_open("X27.251.228.22.106.3.5.59.nc")
print("ncin")
dname <- "hgt"
lon <- ncvar_get(ncin, "lon")
nlon <- dim(lon)
head(lon)
lat <- ncvar_get(ncin, "lat", verbose = F)
nlat <- dim(lat)
head(lat)
print(c(nlon, nlat))
t <- ncvar_get(ncin, "time")
tunits <- ncatt_get(ncin, "time", "units")
nt <- dim(t)
tmp.array <- ncvar_get(ncin, dname)
dlname <- ncatt_get(ncin, dname, "long_name")
dunits <- ncatt_get(ncin, dname, "units")
fillvalue <- ncatt_get(ncin, dname, "_FillValue")
dim(tmp.array)
title <- ncatt_get(ncin, 0, "title")
institution <- ncatt_get(ncin, 0, "institution")
datasource <- ncatt_get(ncin, 0, "source")
references <- ncatt_get(ncin, 0, "references")
history <- ncatt_get(ncin, 0, "history")
Conventions <- ncatt_get(ncin, 0, "Conventions")
# split the time units string into fields
tustr <- strsplit(tunits$value, " ")
tdstr <- strsplit(unlist(tustr)[3], "-")
tmonth = as.integer(unlist(tdstr)[2])
tday = as.integer(unlist(tdstr)[3])
tyear = as.integer(unlist(tdstr)[1])
chron::chron(t, origin = c(tmonth, tday, tyear))
tmp.array[tmp.array == fillvalue$value] <- NA
length(na.omit(as.vector(tmp.array[, , 1])))
m <- 1
tmp.slice <- tmp.array[, , m]
lonlat <- expand.grid(lon, lat)
tmp.vec <- as.vector(tmp.slice)
length(tmp.vec)
tmp.df01 <- data.frame(cbind(lonlat, tmp.vec))
names(tmp.df01) <- c("lon", "lat", paste(dname, as.character(m), sep = "_"))
head(na.omit(tmp.df01), 20)
csvfile <- "cru_tmp_1.csv"
write.table(na.omit(tmp.df01), csvfile, row.names = FALSE, sep = ",")
tmp.vec.long <- as.vector(tmp.array)
length(tmp.vec.long)
tmp.mat <- matrix(tmp.vec.long, nrow = nlon * nlat, ncol = nt)
dim(tmp.mat)
head(na.omit(tmp.mat))
# create a dataframe
lonlat <- expand.grid(lon, lat)
tmp.df02 <- data.frame(cbind(lonlat, tmp.mat))
options(width = 110)
head(na.omit(tmp.df02, 20))
csvfile <- "cru_tmp_2.csv"
write.table(na.omit(tmp.df02), csvfile, row.names = FALSE, sep = ",")

In the case of GCMs, i tried downloading a GCM output. But it took nearly 10 hours for a single variable in my laptop. Is there any requirement for laptops and internet connections to download and use GCM data?

Please give me solution.

With regards
Ashwathi
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on May 09, 2017, 05:07:14 PM
Hai,

In procedure to do statistical downscaling of precipitation, I have selected 8 variables from NCEP-NCAR reanalysis data and have done principal component analysis using XLSTAT. I have got some results after PCA (attached). Based on what value of  the result  will we have to select important variables out of 8 variables? I am totally confused. Please help me.

With regards
Ashwathi 
Title: Re: Statistical downscaling of GCMs
Post by: Alok Pandey on May 09, 2017, 07:54:09 PM
Hello,

Eigenvalues of the covariance matrix of data matrix is used to select the number of newly formed variable (without any physical significance).
To learn more about PCA, go through this educational video :
Lecture series on Stochastic Hydrology by Prof. P. P. Mujumdar, Department of Civil Engineering, IISc Bangalore
https://www.youtube.com/watch?v=TqCnD1avqMY
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on May 25, 2017, 12:04:08 PM
Hai,

In downscaling, i have selected predictor variables such as air temperature at 2m, air temperature at 250Hpa, u wind speed at 250Hpa, V wind speed at 250Hpa, geopotential height at 500Hpa, mean sea level pressure, specific humidity at 2m and vertical wind velocity from NCEP-NCAR reanalysis data. I have standardized the data and done Principal component analysis using XLSTAT. I got 5 principal components. Then from observed station data (predictand) i have deducted mean and then performed multiple regression using XLSTAT.  After regression my correlation is found to be very poor approximately 0.2. Is this acceptable? How can i improve correlation? Is there any mistake in this? Please help me.


With regards
Ashwathi V K
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on May 29, 2017, 04:19:30 PM

Hai

Is there GCMs of finer resolution (1 degree x 1 degree) under CMIP5 project with RCP 2.6 scenario available?

With Regards
Ashwathi V K
Title: Re: Statistical downscaling of GCMs
Post by: Rohith Kannada on June 21, 2017, 08:24:49 PM
Hey

I suggest you, to try change factor or quantile mapping methods to downscale. They are simple and many people are using these methods (am not sure in case of SWAT).
Title: Re: Statistical downscaling of GCMs
Post by: Rohith Kannada on June 21, 2017, 08:31:41 PM

Is there GCMs of finer resolution (1 degree x 1 degree) under CMIP5 project with RCP 2.6 scenario available?

Model CCSM4 provide projections at 0.94*1.25 degree, CMCC-CMs at 0.74*0.75 degree and EC-EARTH provides at 1.11*1.125 degree.
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on June 29, 2017, 10:02:15 AM
Hai,

Thankyou Rohith.

Do you have any material on quantile mapping method of downscaling?


With regards
Ashwathi V K
Title: Re: Statistical downscaling of GCMs
Post by: Rohith Kannada on July 08, 2017, 05:09:08 PM
You will find many materials and papers if you just google.
you can also have look at old discussions in the group about quantile mapping.
just follow the link.
(http://wateradroit.com/forum/index.php/topic,384.msg1269.html#msg1269) (http://wateradroit.com/forum/index.php/topic,979.msg2823.html#msg2823)
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on July 14, 2017, 11:28:08 AM
Hai,

I have downloaded some predictor variables from GFDL climate model of CMIP5 project which is in netcdf format. It has four dimension (lat, lon, time, and pressure levels). I want extract data for one pressure level (500Hpa). Is there any method to extract the values? I have tried with R program. But i have failed. Is there any method. Please help me.


With regards
Ashwathi V K
Title: Re: Statistical downscaling of GCMs
Post by: Rohith Kannada on July 14, 2017, 02:22:58 PM
hey you can use Matlab functions (ncinfo or ncdisp to know the contents and units of the variable in nc file ) and ncread to extract the data.
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on August 10, 2017, 11:11:46 AM
Hai,

While doing statistical downscaling, after selecting predictor variables of all the surrounding grids of the study area, should i take the average of all the grid for each of these variables before doing Principal component Analysis?

Please help me..

With Regards
Ashwathi V K
Title: Re: Statistical downscaling of GCMs
Post by: Alok Pandey on August 11, 2017, 10:00:29 AM
Not needed. After standardization, you can perform PCA.
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on August 30, 2017, 05:25:32 PM
Hai,
I have a netcdf file( GCM data) with 2.5*2 degree resolution which is of 4 dimension (lat, lon, time and pressure level) and  has 1 variable (Example - uwind). I also have Era Interim climate data which is of 2.5*2.5 degree resolution. I want change resolution of GCM data to that of Era Interim data so that points of both the file overlap and then extract data from resampled GCM data for points in between Latitude 7.5 degree N and 15 degree N and Longitude 72.5 and 80 degree East for 500Hpa and 750Hpa pressure levels and convert it to csv file, so that i can use data directly for PCA. Likewise I have to perform these for about 100 files. Is there any way to do this for all the files together? I tried with R program. But I failed. Please help me.

With Regards
Ashwathi V K

Title: Re: Statistical downscaling of GCMs
Post by: Alok Pandey on August 31, 2017, 04:17:57 PM
You can use Grads to regrid and automate regriding for many variables/files.
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on September 28, 2017, 05:34:06 PM
Hai,

I have a doubt on results of PCA regression statistical downscaling of temperature. I have downscaled GFDL- CM3 and MRI-CGCM3 of CMIP5 project for Scenarios RCP2.6, 4.5, 8.5 and found temperature for a period of 2020 to 2100 for different stations.  I have used ERA INTERIM daily data for Calibration. Correlation coefficient of  Calibration and Validation results for different stations ranges between 0.65-0.8.
But in the result it was found that  50% of downscaled temperature data has lesser value for higher RCP. i.e, temperature of RCP 8.5 is lower than that of RCP2.6 for certain days in the period between 2020-2100. I have checked the procedure that i have done. I am not able solve the above mentioned problem.

Can you please suggest any solution for the above mention problem?

With Regards
Ashwathi V K
Title: Re: Statistical downscaling of GCMs
Post by: Rohith Kannada on October 04, 2017, 10:09:24 AM
hi

I think it can be possible in the near future (maybe till 2050s), because rcp2.6 is a "peak and decline" scenario, which reaches the peak around 2050 and declines towards the end of century. While rcp8.5 is raising scenario towards end of the century.

You can refer this
Quote
DOI 10.1007/s10584-011-0148-z

I have not worked much with temperature data, but this is what i feel.

Rohith A N
Title: Re: Statistical downscaling of GCMs
Post by: Alok Pandey on October 05, 2017, 05:38:50 PM
Quote
But in the result it was found that  50% of downscaled temperature data has lesser value for higher RCP. i.e, temperature of RCP 8.5 is lower than that of RCP2.6 for certain days in the period between 2020-2100. I have checked the procedure that i have done. I am not able solve the above mentioned problem.

Can you please suggest any solution for the above mention problem?

My view :
50% of downscaled temperature data has lesser value for higher RCP is "an observation based on analysis performed and not a problem" . Also it doesn't say anything about trends (which may be present and should be look into) in the results obtained. Check for the frequency of events (above certain threshold) obtained in results.
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on October 06, 2017, 11:14:04 AM
Hai Alok and Rohith,

But my doubt is that , for RCP 8.5, most of the temperature  should be greater than that of RCP 2.6 because of higher emission in RCP8.5? And average of daily downscaled temperature for RCP2.6,4.5,8.5 over a period from 2020 to 2100 is excatly same.

With Regards
Ashwathi V K
Title: Re: Statistical downscaling of GCMs
Post by: Alok Pandey on October 18, 2017, 05:08:49 PM
Hi,

The problem you are facing needs thorough look into data used, model created and the results obtained. It will be very difficult for anyone to give satisfactory answer to this problem without checking all the steps followed. I would suggest plotting the raw data of variables of all grid points, bias corrected data, and the downscaled data and then comparing the all the plots for better understanding.
Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on October 25, 2017, 03:02:17 PM
Hai,

Can anybody tell the function in r program to get values of optimum parameters after performing support vector regression?

With regards
Ashwathi V K
Title: Statistical downscaling of GCMs with bias correction
Post by: ASHWATHI V K on December 18, 2017, 04:20:15 PM
Hai,

Please find the following procedure that i have used for downscaling:

Downscaling Method:

GCM- GFDL- CM3, MRI CGCM 3 from CMIP5 Project for RCP2.6, 4.5, 8.5

Predictor variables - Sea level pressure, Surface air temperature, Zonal wind speed at 500Hpa, Zonal wind speed at 850Hpa, Meridonal wind speed at 500Hpa, Meridonal wind speed at 850Hpa, Zonal wind speed at surface, Meridonal wind speed at surface, Geopotential height at 500Hpa, Geopotential height at 850Hpa.

Calibration data - ERA INTERIM Daily data of all the predictor variables.

Procedure-
- Downloaded daily data of above mentioned predictor variables from ERA INTERIM website for a period from 1979
   -2012 (for GFDL - 2.5 degree resolution data and for CGCM3 1.125 degree resolution data). Data include all the
    surrounding grids of study area.
- Extracted data from netcdf files to excel files using R program.Removed Feb 29 data of leap year..
- Standardised all the data of predicted variable for all the grids.
- Done Principal Component Analysis using SPSS software and selected 50 Principal components which give 97% variance.
- Divided data into Calibration and validation datasets.
- Done Multiple linear regression between 50 Principal Components in calibration data set and observed station
 temperature using SPSS statistical software. And found correlation coeficients.
- Using the coefficients from regression during calibration, station temperature was found from 50 Principal Components
  in Validation Period and found Correlation coefficients.
- Calibration and Validation results are attached (PFA).
- Downloaded daily data of above mentioned predictor variables of GFDL- CM3 and CGCM3 model from CMIP5 project
 for three different scenarios for a period from 2020 to 2100.
- Extracted and resampled the data of all the predictor variables of GCM using R program so that final selected grid are
  same as that of ERA INTERIM datasets and converted to excel format. This was done for each scenario for each GCM
  separately.
- Done Standardisation of all the variables
- Done Principal component Analysis of all standardised variables and selected 50 Principal Components which give 97%
  variance
-  Using the coefficients obtained during Multiple linear regression during Calibration, temperature for each station was
   found.

After following above procedure if am not getting a proper downscaled value. I wanted to check how my result changes when i include bias correction.

How can i include bias correction in the above procedure? can i apply bias correction directly to the downscaled result??
Please help me.


With Regards
Ashwathi V K

Title: Re: Statistical downscaling of GCMs
Post by: ASHWATHI V K on February 20, 2018, 04:30:08 PM
Hai,

Can any one explain the procedure to do "change factor method of statistical downscaling"?


With Regards
Ashwathi