Hao.Jia
文章
59
分类
15
评论
0

DolphinDB机器学习在金融行业的应用：波动率预测

| DolphinDB •
字数总计 28765 | 阅读时长 71分钟 | 阅读量 629

DolphinDB机器学习在金融行业的应用：波动率预测

波动率是金融资产价格的波动程度，是对资产收益率不确定性的衡量，用于反映金融资产的风险水平。波动率越高，金融资产价格的波动越剧烈，资产收益率的不确定性就越强；波动率越低，金融资产价格的波动越平缓，资产收益率的确定性就越强。

无论是对即将正式进入股指期货市场的各类参与者还是我国的金融监管当局而言, 对股指期货的波动特征和风险状况进行准确的刻画和科学的预测, 进而探索有效的期指市场风险防范和监控手段, 都具有非常重要的理论和现实意义。[1]

DolphinDB是由浙江智臾科技有限公司研发的一款 高性能分布式时序数据库 ，集成了功能强大的编程语言和高容量高速度的流数据分析系统，为海量结构化数据的快速存储、检索、分析及计算提供一站式解决方案，适用于量化金融及工业物联网等领域。

DolphinDB特别适用于对速度要求极高的低延时或实时性任务，如基于 海量历史数据 的交互式查询与计算、高频交易策略研发与实施、实时数据处理与监控等。其查询和计算速度，是常用大数据系统的10到1000倍。DolphinDB提供一站式解决方案，用户可在同一个系统内使用简洁高效的脚本语言快速开发大数据应用。与市场同类系统相比，DolphinDB凭借其卓越的技术优势来提高数据分析人员的用户体验，显著缩短数据研究和分析的周期，节约企业使用数据的人力和物力成本，帮助企业快速实施基于大数据的产品研发，使其在激烈的竞争中脱颖而出。

本文将利用DolphinDB的内置机器学习算法，以及上海证券交易所2020一年的snapshot数据，实现股票市场未来10分钟的波动率预测。

项目运行环境：

OS：64位 CentOS Linux release 7.9.2009 (Core)

物理 CPU 个数：2

每个物理 CPU 中 core 的个数：10

超线程数：2

逻辑 CPU 个数：40

内存：520G

hdd盘：2.2T * 10块

1 Snapshot数据文件结构说明

证券快照：每幅快照间隔为 3 秒或 5 秒（相邻快照如果完全相同就保留第一幅），将一天所有证券的快照按时间顺序存于一个文件中；





共174个字段

2 数据预处理

2.1 数据选择

本项目用到的字段为Snapshot中的部分字段有：
股票代码，快照时间，申买十价，申买十量，申卖十价，申卖十量

SecurityID,TradeTime,BidPrice0,BidPrice1,BidPrice2,BidPrice3,BidPrice4,BidPrice5,BidPrice6,BidPrice7,BidPrice8,BidPrice9,BidOrderQty0,BidOrderQty1,BidOrderQty2,BidOrderQty3,BidOrderQty4,BidOrderQty5,BidOrderQty6,BidOrderQty7,BidOrderQty8,BidOrderQty9,OfferPrice0,OfferPrice1,OfferPrice2,OfferPrice3,OfferPrice4,OfferPrice5,OfferPrice6,OfferPrice7,OfferPrice8,OfferPrice9,OfferOrderQty0,OfferOrderQty1,OfferOrderQty2,OfferOrderQty3,OfferOrderQty4,OfferOrderQty5,OfferOrderQty6,OfferOrderQty7,OfferOrderQty8,OfferOrderQty9

样本为2020年上证50指数的成分股：

股票代码：601318,600519,600036,600276,601166,600030,600887,600016,601328,601288,600000,600585,601398,600031,601668,600048,601888,600837,601601,601012,603259,601688,600309,601988,601211,600009,600104,600690,601818,600703,600028,601088,600050,601628,601857,601186,600547,601989,601336,600196,603993,601138,601066,601236,601319,603160,600588,601816,601658,600745

股票名称：中国平安，贵州茅台，招商银行，恒瑞医药，兴业银行，中信证券，伊利股份，民生银行，交通银行，农业银行，浦发银行，海螺水泥，工商银行，三一重工，中国建筑，保利地产，中国中免，海通证券，中国太保，隆基股份，药明康德，华泰证券，万华化学，中国银行，国泰君安，上海机场，上汽集团，海尔智家，光大银行，三安光电，中国石化，中国神华，中国联通，中国人寿，中国石油，中国铁建，山东黄金，中国重工，新华保险，复星医药，洛阳钼业，工业富联，中信建投，红塔证券，中国人保，汇顶科技，用友网络，京沪高铁，邮储银行，闻泰科技

Snapshot数据已经提前导入至DolphinDB中，导入方法见股票行情数据导入实例

2.2 数据获取

三秒一张的快照数据量是非常巨大的，如果获取的股票数量较多，则会占用大量内存，但是经过数据预处理（数据清洗，特征工程，重采样）之后数据量很少，因此可以采用分批处理的方式，每次只选取固定数量的股票，上一批处理完之后再处理下一批，处理完成的数据存在table中。

//数据库连接并加载snapshot的table

dbName="dfs://snapshot_SH_L2_OLAP"
tableName="snapshot_SH_L2_OLAP"
db = database(dbName)
snapshot=loadTable(db,tableName)

//stock_list 保存用于该项目的股票代码向量
stock_list=`601318`600519`600036`600276`601166`600030`600887`600016`601328`601288`600000`600585`601398`600031`601668`600048`601888`600837`601601`601012`603259`601688`600309`601988`601211`600009`600104`600690`601818`600703`600028`601088`600050`601628`601857`601186`600547`601989`601336`600196`603993`601138`601066`601236`601319`603160`600588`601816`601658`600745

//分批操作，本项目中将50支股票拆分为每25支/批进行处理，该数值可以根据硬件配置随意调整
security_id_epochs=cut(stock_list,25);

//dataset 保存处理完成后的数据，各字段含义见下文
dataset = table(1000000:0 , `interval_TradeTime`SecurityID`BAS`DI0`DI1`DI2`DI3`DI4`DI5`DI6`DI7`DI8`DI9`Press`RV,`TIMESTAMP`SYMBOL`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE)

//读取数据 筛选了开盘时段的数据（9:30-11:30 13:00-15:00）
for(security_id in security_id_epochs)
{
    order_book = select SecurityID,TradeTime,BidPrice0,BidPrice1,BidPrice2,BidPrice3,BidPrice4,BidPrice5,BidPrice6,BidPrice7,BidPrice8,BidPrice9,BidOrderQty0,BidOrderQty1,BidOrderQty2,BidOrderQty3,BidOrderQty4,BidOrderQty5,BidOrderQty6,BidOrderQty7,BidOrderQty8,BidOrderQty9,OfferPrice0,OfferPrice1,OfferPrice2,OfferPrice3,OfferPrice4,OfferPrice5,OfferPrice6,OfferPrice7,OfferPrice8,OfferPrice9,OfferOrderQty0,OfferOrderQty1,OfferOrderQty2,OfferOrderQty3,OfferOrderQty4,OfferOrderQty5,OfferOrderQty6,OfferOrderQty7,OfferOrderQty8,OfferOrderQty9 from snapshot where (TradeTime.minute()>=09:30:00 and  TradeTime.minute()<=11:30:00 and SecurityID in security_id) or  (TradeTime.minute()>=13:00:00 and  TradeTime.minute()<=15:00:00 and SecurityID in security_id)

    //特征工程代码见下文
}

2.3 特征工程

参考文献[2][3][4][5]

bid/ask spread(BAS):用于衡量买单价和卖单价的价差

//bid/ask spread(BAS):用于表示买价和卖价的价差
order_book[`BAS] = (order_book[`OfferPrice0]\order_book[`BidPrice0])-1

Weighted averaged price(WAP)：加权平均价格

//Weighted averaged price(WAP)：加权平均价格
order_book[`WAP] = (order_book[`BidPrice0]*order_book[`OfferOrderQty0] + order_book[`OfferPrice0]*order_book[`BidOrderQty0])\(order_book[`BidOrderQty0]+order_book[`OfferOrderQty0])

Depth Imbalance(DI):深度不平衡

//Depth Imbalance(DI):深度不平衡
for( i in 0:10 )
{
	order_book[`DI +i]=(order_book[`BidOrderQty +i]-order_book[`OfferOrderQty +i])\(order_book[`BidOrderQty +i]+order_book[`OfferOrderQty +i])
}

Press：买卖压力指标

//Press：买卖压力指标
for( i in 0:10 )
{
	order_book["Bid_1_P_WAP"+i]=1\(order_book[`BidPrice +i]-order_book[`WAP])
}
for( i in 0:10 )
{
	order_book["Offer_1_P_WAP"+i]=1\(order_book[`OfferPrice +i]-order_book[`WAP])
}
order_book[`Bid_1_P_WAP_SUM]=order_book["Bid_1_P_WAP0"]+order_book["Bid_1_P_WAP1"]+order_book["Bid_1_P_WAP2"]+order_book["Bid_1_P_WAP3"]+order_book["Bid_1_P_WAP4"]+order_book["Bid_1_P_WAP5"]+order_book["Bid_1_P_WAP6"]+order_book["Bid_1_P_WAP7"]+order_book["Bid_1_P_WAP8"]+order_book["Bid_1_P_WAP9"]

order_book[`Offer_1_P_WAP_SUM] = order_book["Offer_1_P_WAP0"]+order_book["Offer_1_P_WAP1"]+order_book["Offer_1_P_WAP2"]+order_book["Offer_1_P_WAP3"]+order_book["Offer_1_P_WAP4"]+order_book["Offer_1_P_WAP5"]+order_book["Offer_1_P_WAP6"]+order_book["Offer_1_P_WAP7"]+order_book["Offer_1_P_WAP8"]+order_book["Offer_1_P_WAP9"]

order_book[`BidPress]=order_book[`BidOrderQty0]*(order_book["Bid_1_P_WAP0"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty1]*(order_book["Bid_1_P_WAP1"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty2]*(order_book["Bid_1_P_WAP2"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty3]*(order_book["Bid_1_P_WAP3"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty4]*(order_book["Bid_1_P_WAP4"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty5]*(order_book["Bid_1_P_WAP5"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty6]*(order_book["Bid_1_P_WAP6"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty7]*(order_book["Bid_1_P_WAP7"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty8]*(order_book["Bid_1_P_WAP8"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty9]*(order_book["Bid_1_P_WAP9"]\order_book[`Bid_1_P_WAP_SUM])

order_book[`OfferPress]=order_book[`OfferOrderQty0]*(order_book["Offer_1_P_WAP0"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty1]*(order_book["Offer_1_P_WAP1"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty2]*(order_book["Offer_1_P_WAP2"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty3]*(order_book["Offer_1_P_WAP3"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty4]*(order_book["Offer_1_P_WAP4"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty5]*(order_book["Offer_1_P_WAP5"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty6]*(order_book["Offer_1_P_WAP6"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty7]*(order_book["Offer_1_P_WAP7"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty8]*(order_book["Offer_1_P_WAP8"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty9]*(order_book["Offer_1_P_WAP9"]\order_book[`Offer_1_P_WAP_SUM])

order_book[`Press]= log(order_book[`BidPress])-log(order_book[`OfferPress])

特征数据重采样（10min窗口，并聚合计算波动率）

重采样利用group by SecurityID, interval( TradeTime, 10m, "none" )方法

Realized volatility(RV)：波动率定义，对数收益率的平方和的平方根

股票的价格始终是处于买单价和卖单价之间，因此本项目用加权平均价格来代替股价进行计算

//自定义的聚合函数，用于计算波动率
def RV(WAP)
{
	LogWAP = log(WAP)
	LogReturn = LogWAP-prev(LogWAP)
	return sqrt(sum(LogReturn*LogReturn))
}
//将预处理后的数据加载至dataset表中
append!(dataset,select SecurityID,mean(BAS) as BAS,mean(DI0) as DI0,mean(DI1) as DI1,mean(DI2) as DI2,mean(DI3) as DI3,mean(DI4) as DI4,mean(DI5) as DI5,mean(DI6) as DI6,mean(DI7) as DI7,mean(DI8) as DI8,mean(DI9) as DI9,mean(Press) as Press,RV(WAP) as RV from order_book group by SecurityID, interval( TradeTime, 10m, "none" ))

数据预处理时间：3m 1s 208ms

3 构建模型

机器学习模型选择adaBoostRegressor

评价指标：根均方百分比误差(Root Mean Square Percentage Error, RMSPE)

3.1 建立训练集和测试集

本项目中没有设置验证集，训练集测试集划分： train:test = 189963:81413

def trainTestSplit(x, testRatio) {
    xSize = x.size()
    testSize =( xSize * (1-testRatio))$INT
    return x[0: testSize], x[testSize:xSize]
}
Train, Test = trainTestSplit(dataset, 0.3)

3.2 训练及评价

def RMSPE(a,b)
{
	return sqrt( sum( ((a-b)\a)*((a-b)\a) ) \a.size()  )
}


model = adaBoostRegressor(sqlDS(<select * from Train>),yColName=`targetRV,xColNames=`BAS`DI0`DI1`DI2`DI3`DI4`Press,numTrees=60,maxDepth=20,loss=`square)
predicted = model.predict(Test)
Test[`predict]=predicted
print("RMSPE="+RMSPE(Test.targetRV,predicted))

RMSPE=10.5469
运行时间：1m 24s 303ms

调参记录表

该项目采用手动粗调参，不代表模型及应用的最优结果

RMSPE time numTrees maxDepth features
20.8606 3m 12s 873ms 100 20 BAS,DI0-9,Press,RV
23.4696 1m 28s 389ms 50 20 BAS,DI0-9,Press,RV
17.4161 2m 35s 670ms 100 20 BAS,DI0-4,Press,RV
29.7012 2m 26s 81ms 100 20 BAS,DI0-3,Press,RV
20.6451 2m 25s 296ms 100 20 BAS,DI0-4,RV
17.2691 2m 16s 781ms 100 20 BAS,DI0-4,Press
17.1232 2m 20s 427ms 100 20 BAS,DI0-4,Press
15.2243 1m 53s 989ms 80 20 BAS,DI0-4,Press
10.5469* 1m 24s 303ms 60 20 BAS,DI0-4,Press
23.6367 57s 858ms 40 20 BAS,DI0-4,Press

4 结果数据可视化

该部分将Test表导出至本地，利用python进行数据可视化，展示测试集的拟合效果

import pandas as pd
import numpy as np
import plotly.express as px
import random
test = pd.read_csv("output.csv") # 到处的csv文件地址
stock_id = test['SecurityID'].unique()
stock_id
length=200
start=200
i=random.randint(0,len(stock_id))
data = test[test['SecurityID']==stock_id[i]]
data_sample=data[start:start+length]
data_sample['time_id']=np.array(range(start,start+length))
fig = px.line(data_sample, x="time_id", y=["targetRV","predict"], title=f'target and prediction of SecurityID =={stock_id[i]}')
fig.show()

蓝色线条为真实值

红色线条为预测值

中国平安[601318] 部分波动率真实预测

工业富联[601138] 部分波动率真实预测

海通证券[600837] 部分波动率真实预测

山东黄金[600547] 部分波动率真实预测

中信建投[601066] 部分波动率真实预测

药明康德[603259] 部分波动率真实预测

海尔智家[600690] 部分波动率真实预测

中国太保[601601] 部分波动率真实预测

汇顶科技[603160] 部分波动率真实预测

中国中免[601888] 部分波动率真实预测

5 参考文献

[1]郑振龙, 汤文玉. 波动率风险及风险价格——来自中国A股市场的证据[J]. 金融研究编辑部, 2011.

[2]Introduction to financial concepts and data

[3]Daivy. 【高频】Level-2高频数据的实证研究（一）

[4]临客. Optiver Realized Volatility Prediction

[5]Daivy. 【高频】基于GBDT-FM模型的level-2高频数据实证研究（二）

6 附录

数据预处理完整代码

login("admin","123456");
clearAllCache();
undef(all);
go;

stock_list=`601318`600519`600036`600276`601166`600030`600887`600016`601328`601288`600000`600585`601398`600031`601668`600048`601888`600837`601601`601012`603259`601688`600309`601988`601211`600009`600104`600690`601818`600703`600028`601088`600050`601628`601857`601186`600547`601989`601336`600196`603993`601138`601066`601236`601319`603160`600588`601816`601658`600745

dbName="dfs://snapshot_SH_L2_OLAP"
tableName="snapshot_SH_L2_OLAP"
db = database(dbName)
snapshot=loadTable(db,tableName)
def RV(WAP)
{
	LogWAP = log(WAP)
	LogReturn = LogWAP-prev(LogWAP)
	return sqrt(sum(LogReturn*LogReturn))
}

//分片读取
//security_id_list = (select distinct(SecurityID) from snapshot).distinct_SecurityID;
security_id_epochs=cut(stock_list,25);
dataset = table(1000000:0 , `interval_TradeTime`SecurityID`BAS`DI0`DI1`DI2`DI3`DI4`DI5`DI6`DI7`DI8`DI9`Press`RV,`TIMESTAMP`SYMBOL`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE)
for(security_id in security_id_epochs)
{
	order_book = select SecurityID,TradeTime,BidPrice0,BidPrice1,BidPrice2,BidPrice3,BidPrice4,BidPrice5,BidPrice6,BidPrice7,BidPrice8,BidPrice9,BidOrderQty0,BidOrderQty1,BidOrderQty2,BidOrderQty3,BidOrderQty4,BidOrderQty5,BidOrderQty6,BidOrderQty7,BidOrderQty8,BidOrderQty9,OfferPrice0,OfferPrice1,OfferPrice2,OfferPrice3,OfferPrice4,OfferPrice5,OfferPrice6,OfferPrice7,OfferPrice8,OfferPrice9,OfferOrderQty0,OfferOrderQty1,OfferOrderQty2,OfferOrderQty3,OfferOrderQty4,OfferOrderQty5,OfferOrderQty6,OfferOrderQty7,OfferOrderQty8,OfferOrderQty9 from snapshot where (TradeTime.minute()>=09:30:00 and  TradeTime.minute()<=11:30:00 and SecurityID in security_id) or  (TradeTime.minute()>=13:00:00 and  TradeTime.minute()<=15:00:00 and SecurityID in security_id)
	//bid/ask spread(BAS):用于表示买价和卖价的价差
	order_book[`BAS] = (order_book[`OfferPrice0]\order_book[`BidPrice0])-1
	//Weighted averaged price(WAP)：加权平均价格
	order_book[`WAP] = (order_book[`BidPrice0]*order_book[`OfferOrderQty0] + order_book[`OfferPrice0]*order_book[`BidOrderQty0])\(order_book[`BidOrderQty0]+order_book[`OfferOrderQty0])
	//Depth Imbalance(DI):深度不平衡
	for( i in 0:10 )
	{
		order_book[`DI +i]=(order_book[`BidOrderQty +i]-order_book[`OfferOrderQty +i])\(order_book[`BidOrderQty +i]+order_book[`OfferOrderQty +i])
	}
	//Press：买卖压力指标
	for( i in 0:10 )
	{
		order_book["Bid_1_P_WAP"+i]=1\(order_book[`BidPrice +i]-order_book[`WAP])
	}
	for( i in 0:10 )
	{
		order_book["Offer_1_P_WAP"+i]=1\(order_book[`OfferPrice +i]-order_book[`WAP])
	}
	order_book[`Bid_1_P_WAP_SUM]=order_book["Bid_1_P_WAP0"]+order_book["Bid_1_P_WAP1"]+order_book["Bid_1_P_WAP2"]+order_book["Bid_1_P_WAP3"]+order_book["Bid_1_P_WAP4"]+order_book["Bid_1_P_WAP5"]+order_book["Bid_1_P_WAP6"]+order_book["Bid_1_P_WAP7"]+order_book["Bid_1_P_WAP8"]+order_book["Bid_1_P_WAP9"]
	
	order_book[`Offer_1_P_WAP_SUM] = order_book["Offer_1_P_WAP0"]+order_book["Offer_1_P_WAP1"]+order_book["Offer_1_P_WAP2"]+order_book["Offer_1_P_WAP3"]+order_book["Offer_1_P_WAP4"]+order_book["Offer_1_P_WAP5"]+order_book["Offer_1_P_WAP6"]+order_book["Offer_1_P_WAP7"]+order_book["Offer_1_P_WAP8"]+order_book["Offer_1_P_WAP9"]
	
	order_book[`BidPress]=order_book[`BidOrderQty0]*(order_book["Bid_1_P_WAP0"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty1]*(order_book["Bid_1_P_WAP1"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty2]*(order_book["Bid_1_P_WAP2"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty3]*(order_book["Bid_1_P_WAP3"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty4]*(order_book["Bid_1_P_WAP4"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty5]*(order_book["Bid_1_P_WAP5"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty6]*(order_book["Bid_1_P_WAP6"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty7]*(order_book["Bid_1_P_WAP7"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty8]*(order_book["Bid_1_P_WAP8"]\order_book[`Bid_1_P_WAP_SUM])+order_book[`BidOrderQty9]*(order_book["Bid_1_P_WAP9"]\order_book[`Bid_1_P_WAP_SUM])
	
	order_book[`OfferPress]=order_book[`OfferOrderQty0]*(order_book["Offer_1_P_WAP0"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty1]*(order_book["Offer_1_P_WAP1"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty2]*(order_book["Offer_1_P_WAP2"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty3]*(order_book["Offer_1_P_WAP3"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty4]*(order_book["Offer_1_P_WAP4"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty5]*(order_book["Offer_1_P_WAP5"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty6]*(order_book["Offer_1_P_WAP6"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty7]*(order_book["Offer_1_P_WAP7"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty8]*(order_book["Offer_1_P_WAP8"]\order_book[`Offer_1_P_WAP_SUM])+order_book[`OfferOrderQty9]*(order_book["Offer_1_P_WAP9"]\order_book[`Offer_1_P_WAP_SUM])
	
	order_book[`Press]= log(order_book[`BidPress])-log(order_book[`OfferPress])
	
	append!(dataset,select SecurityID,mean(BAS) as BAS,mean(DI0) as DI0,mean(DI1) as DI1,mean(DI2) as DI2,mean(DI3) as DI3,mean(DI4) as DI4,mean(DI5) as DI5,mean(DI6) as DI6,mean(DI7) as DI7,mean(DI8) as DI8,mean(DI9) as DI9,mean(Press) as Press,RV(WAP) as RV from order_book group by SecurityID, interval( TradeTime, 10m, "none" ))
}

dataset[`targetRV]=next(dataset[`RV])
dataset=dataset[each(isValid, dataset.values()).rowAnd()] // drop na

dataset保存至dolphindb的代码

login("admin", "123456")
dbName = "dfs://SZ50VolatityDataSet"
tbName = "SZ50VolatityDataSet"
db = database(dbName, VALUE, 2020.01.01..2020.12.31)
name = `interval_TradeTime`SecurityID`BAS`DI0`DI1`DI2`DI3`DI4`DI5`DI6`DI7`DI8`DI9`Press`RV`targetRV
type=`TIMESTAMP`SYMBOL`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE`DOUBLE
tbTemp = table(1:0, name, type)
db = database(dbName)
createPartitionedTable(dbHandle=db, table=tbTemp, tableName=tbName, partitionColumns=`interval_TradeTime, compressMethods={TradeTime:"delta"})
snapshot_SH_L2_Volatity_DataSet = loadTable(dbName, tbName)
append!(snapshot_SH_L2_Volatity_DataSet,dataset)

构建模型的代码

login("admin","123456");
clearAllCache();
undef(all);
go;

//dataset读取
login("admin", "123456")
dbName = "dfs://SZ50VolatityDataSet"
tbName = "SZ50VolatityDataSet"
dataset = select * from loadTable(dbName, tbName)
//训练
def trainTestSplit(x, testRatio) {
    xSize = x.size()
    testSize =( xSize * (1-testRatio))$INT
    return x[0: testSize], x[testSize:xSize]
}
Train, Test = trainTestSplit(dataset, 0.3)
def RMSPE(a,b)
{
	return sqrt( sum( ((a-b)\a)*((a-b)\a) ) \a.size()  )
}

model = adaBoostRegressor(sqlDS(<select * from Train>),yColName=`targetRV,xColNames=`BAS`DI0`DI1`DI2`DI3`DI4`Press,numTrees=60,maxDepth=20,loss=`square)
predicted = model.predict(Test)
Test[`predict]=predicted
print("RMSPE="+RMSPE(Test.targetRV,predicted))

数据可视化代码

import pandas as pd
import pandas as pd
import numpy as np
import plotly.express as px
import random
test = pd.read_csv("output1.csv") # 导出数据的路径
stock_id = test['SecurityID'].unique()
length=100 # 展示部分数据的长度
start=200 # 展示部分数据的开始位置
i=random.randint(0,len(stock_id)) # 随机选择一支股票进行展示
data = test[test['SecurityID']==stock_id[i]]
data_sample=data[start:start+length]
data_sample['time_id']=np.array(range(start,start+length))
fig = px.line(data_sample, x="time_id", y=["targetRV","predict"], title=f'target and prediction of SecurityID =={stock_id[i]}')
fig.show()
文章作者： Hao.Jia
本文链接：
版权声明： 本站所有文章除特别声明外，均采用 CC BY-NC-SA 4.0 许可协议。转载请注明来自 Hao.Jia's Blog！
DolphinDB 机器学习 dolphindb
喜欢就支持一下吧