Real Time Data Now / Nest Trader to Amibroker, Fcharts

Snake.Head

Well-Known Member
@josh1
This working properly with RTD 1.25

Auto script update from marketwatch Bug
update RTD/Preserve in new RTD 1.26/1.30
Take two scene here
If settings under update RTD/Preserve Yes and no to 8
What it dose't take script from market watch but give error "dude at least one script selected"
Note .ini is update with script from user marketwatch.

If i go with workaround you mentioned in few post back
First open backfill and select script from backfill windows and start rtd
it throw same here
where in you look into settings.in it update script in .ini
script here update but no. has it beyond 8 script so rtd throw same error "dude at least one script selected"

Scrip9=nse_cm|RELCAPITAL-EQ;RELCAPITAL;LTP;LTT;Volume Traded Today;Open Interest
Scrip10=nse_cm|TATASTEEL-EQ;TATASTEEL;LTP;LTT;Volume Traded Today;Open Interest
Scrip11=nse_fo|NIFTY15AUGFUT;NIFTY_F1;LTP;LTT;Volume Traded Today;Open Interest
Scrip12=mcx_fo|CRUDEOILM15AUGFUT;CRUDEOILM_F1;LTP;LTT;Volume Traded Today;Open Interest
 

josh1

Well-Known Member
@josh1
This working properly with RTD 1.25

Auto script update from marketwatch Bug
update RTD/Preserve in new RTD 1.26/1.30
Take two scene here
If settings under update RTD/Preserve Yes and no to 8
What it dose't take script from market watch but give error "dude at least one script selected"
Note .ini is update with script from user marketwatch.

If i go with workaround you mentioned in few post back
First open backfill and select script from backfill windows and start rtd
it throw same here
where in you look into settings.in it update script in .ini
script here update but no. has it beyond 8 script so rtd throw same error "dude at least one script selected"
Use RTD_1.3. Uploaded now. This error removed.
 

rmike

Well-Known Member
Josh said:
RTDMan does not do backfill. RTD1.26 does it
I meant RTD 1.26 tested with both versions of RTDMan (bundled & Mod) failed to accomplish backfill
Josh said:
Temp output files will be blank if you have not arranged columns in Data table or Hourly statistics as prescribed in tutorial
No they were arranged as per the help file
Josh said:
Delete tics launches separate application. It picks up all scrips from Amibroker database and not from marketwatch
Josh said:
RTD 1.26 includes scrips in settings.ini if you select scrips from its GUI (Backfill)
Ah! Wasn't aware of these nuances as was using the GUI version for the first time. As a suggestion, this info could be included in the help file too
Josh said:
Josh1 and Shankar are still amateur programmers.
40 global variables including one array were declared inside functions. Huge memory leak was guaranteed
:) That's a given! It doesn't lessen the appreciation, though! :) However, an app has to be tested on its own merits objectively, keeping the relationship with the developer aside :). Its human to not find adverse results very likeable, but testing would lose purpose if facts were not reported plainly
Josh said:
Live and learn all the time
Right said Fred!!! :)
Josh said:
Created revised installer and uploaded to dropbox. I am calling it RTD_1.30. If you get time please check for memory leaks once again
Thanks, downloading now. May not be able to immediately attend to it as am a bit tied down for a couple of days. Will get back once done

Regards,
 

TracerBullet

Well-Known Member
Memory - 20mb is nothing. It does not imply a leak. If there is a suspected leak, you need to test if the memory usage increases with time without reason.
So a normally 20 mb application that suddenly starts using 400 mb - thats a leak. Or if it keeps increasing - 20 30 40 50 ....
Memory leak happens if you keep on adding a resource without releasing it. This is more of a problem in c/c++. One example in autoit that i can guess would be to keep on adding global variables (dynamically?) which only get released on application exit. Just manually using 100 variables is not a leak, but maybe not necessary.

Also, i again say it - if not already done, when you get time, you can try to do 1 click backfills. Just set up scrips once and then subsequent runs should backfill automatically in background. this makes the backfill process painless and automatic.
 

josh1

Well-Known Member
Memory - 20mb is nothing. It does not imply a leak. If there is a suspected leak, you need to test if the memory usage increases with time without reason.
So a normally 20 mb application that suddenly starts using 400 mb - thats a leak. Or if it keeps increasing - 20 30 40 50 ....
Memory leak happens if you keep on adding a resource without releasing it. This is more of a problem in c/c++. One example in autoit that i can guess would be to keep on adding global variables (dynamically?) which only get released on application exit. Just manually using 100 variables is not a leak, but maybe not necessary.

Also, i again say it - if not already done, when you get time, you can try to do 1 click backfills. Just set up scrips once and then subsequent runs should backfill automatically in background. this makes the backfill process painless and automatic.
Declaring global variable in function redefines it when the function is invoked again. That creates memory leak in Autoit.

I don't understand what you want. Right now the process is 3 clicks. Click on backfill -- Select Source NEST Plus or VWAP/Stat or Index -- Select All -- Do Backfill. Is to so painful? In any case backfill cannot be done without NOW and desired marketwatch open.
With new code RTDMan runs in tandem with Terminal. There is no need to backfill if there is no network disconnection.
 

TracerBullet

Well-Known Member
Declaring global variable in function redefines it when the function is invoked again. That creates memory leak in Autoit.
.
ok

I don't understand what you want. Right now the process is 3 clicks. Click on backfill -- Select Source NEST Plus or VWAP/Stat or Index -- Select All -- Do Backfill. Is to so painful? In any case backfill cannot be done without NOW and desired marketwatch open.
With new code RTDMan runs in tandem with Terminal. There is no need to backfill if there is no network disconnection.
If you are happy with it, no problem. But again, Backfill can be done without NOW fronted ( but with market watch open for NestPlus - not needed for VWAP). All i do is send keystrokes and control clicks in AHK.

I prefer to select stuff once and then only need to execute to do backfill in background. RTDMan is close but it cannot be same as NOW server by design.
And in my case, although NOW works very well, my internet is shared. One blind click and forget is much easier that multiple clicks with choose.

To give you some idea, here is an early code version. There are many things mssing so this is not a RTD replacement, but it is usable enough for me. It is NOW only. Also, for now i just ignore volume and overwrite the bar in Amibroker. ABBackfill also has some changes so below is only for view.

Code:
#CommentFlag // 

SendMode Input  				// Recommended for new scripts
SetWorkingDir %A_ScriptDir%  	// Ensures a consistent starting directory.
SetTitleMatchMode, 2 			// A window's title can contain the text anywhere
//!F1:: ; Press Alt F1 to execute

// TODO - ini to variable + split up stuff
// TODO - ABBackFIll error handling - Put Time + append as log 


// TODO - DataTable - Get only todays data 
// TODO - VWAP - HS needs extra columns for volume	- Just select correct columns 
// TODO - Index Backfill - Space in name

// TODO - DataTable - Search columns for Trading symbol, remove TradingSymbolColIndex 
// TODO - Datatable - Loop Scrips 
// TODO - VWAP - Better logic to check if vwap data has been fetched 
// TODO - VWAP loop scrips 


// ------------------------- Settings ----------------------------- 
 
scripAlias 		= NIFTY_F1
mode			= DT							// DT or VWAP
nowWindowTitle	= NOW 1.13
backfillExePath = %A_ScriptDir%\ABBackFill
backfillExe		= ABBackFill\ABBackFill.exe

// ---------- VWAP -----------

param1 = NFO		 							// Exchg-Seg
param2 = FUTIDX		 							// Series
param3 = NIFTY		 							// Symbol
param4 = 1			 							// Expiry Date - Set by Position
param5 = XX										// Opt Type
param6 =		 	 							// Strike Price

VWAPSleepTime = 5000 							// Wait Time in ms

vwapWindowTitle		 = Hourly Statistics
VWAPBackfillFileName = ABBackFill\VWAP.txt

// ------- DataTable ---------

TradingSymbol 		  = NIFTY15AUGFUT
TradingSymbolColIndex = 2													// Position of Trading Symbol Column in Market Watch
dtWindowTitle		  = DataTable
DTBackfillFileName    = ABBackFill\DataTable.txt

// ----------------------------------------------------------------


IfWinExist, NOW 1.13
{	
	global mode
	
	if( mode = "DT" ){
		dtBackFill()
	}
	else{	
		vwapBackFill()
	}	
}
return

// -------------------------------- DT -----------------------------------------------------

// NestPlus DataTable Backfill
dtBackFill()
{
	global backfillExe, backfillExePath, dtWindowTitle
	
	openDataTable()
	writeDTData()		
		
	Run, %backfillExe%, %backfillExePath%, hide								// Do backfill by calling exe	
	WinClose, %dtWindowTitle%,, 0 											// Close DT - waits 500ms
}
openDataTable()
{
	global nowWindowTitle, dtWindowTitle, TradingSymbolColIndex, TradingSymbol
	
	ControlGet, RowCount, List, Count, SysListView323, %nowWindowTitle%		// No of rows in MarketWatch	
	ControlSend, SysListView323, {Home}, %nowWindowTitle%					// Start from top and search for scrip
			
	Loop, %RowCount%{														// Select row with our scrip		
		ControlGet, symbol, List, Selected Col%TradingSymbolColIndex%, SysListView323, %nowWindowTitle%							
																			// Take Trading Symbol from column Number in %TradingSymbolColIndex%
		if( symbol = TradingSymbol ){										// and compare it with %TradingSymbol%
			 break
		}
		ControlSend, SysListView323, {Down}, %nowWindowTitle%				// Move Down to next row if not found yet
	}	
	
	if ( symbol != TradingSymbol ) {
		MsgBox, %TradingSymbol% Not Found
		ExitApp
	}
	
	ControlSend, SysListView323, {Shift}D, %nowWindowTitle%					// At this point row should be selected. Open Data Table with shift-d	
	WinWait     %dtWindowTitle%,, 30										// Wait for Data Table to open
	WinMinimize %dtWindowTitle%
}

// columns expected order - TradingSymbol Time O H L C V
writeDTData()
{
	global dtWindowTitle, DTBackfillFileName, VWAPBackfillFileName, scripAlias
	
	ControlGet, data, List, , SysListView321, %dtWindowTitle%				// Copy Data
	
	IfExist, %VWAPBackfillFileName%											// Clear Out VWAP data
	{
		file := FileOpen(VWAPBackfillFileName, "w" )	  
		if IsObject(file){
			file.Write(" ")
			file.Close()
		}
	}
	
	IfExist, %DTBackfillFileName%
	{				
		file := FileOpen(DTBackfillFileName, "w" )	   			    		// := does not need %% for var
		if !IsObject(file){
			MsgBox "Can't open DT backfill file for writing"
			ExitApp
		}
		
		AliasText := "name=" . scripAlias . "`n"
		file.Write(AliasText)												// Append AB Scrip Name		
		file.Write(data)							    					// Add Data
		file.Close()		
	}
	else{
		MsgBox "DT backfill file Not Found" 
		ExitApp
	}
}

// ------------------------------- VWAP ----------------------------------------------------

// Open Hourly Stats 
vwapBackFill()
{
	global backfillExe, backfillExePath, vwapWindowTitle
		
	openVwap()	
	writeVwapData()															// Write csv 
	
	Run, %backfillExe%, %backfillExePath%, hide								// Do backfill by calling exe		
	WinClose, %vwapWindowTitle%,, 0 										// Close HS - waits 500ms	
}

openVwap(){
	global nowWindowTitle, vwapWindowTitle, param1,param2,param3,param4,param5,param6
	
	WinMenuSelectItem, %nowWindowTitle%,, Market, Hourly Statistics			// Open HS using NOW Menu
	WinWait, %vwapWindowTitle%,,10											// Wait for HS to open
	WinMinimize %vwapWindowTitle%
	
	Control, ChooseString , %param1%, ComboBox1, %vwapWindowTitle%			// Set Params - Exchg-Seg	
	Control, ChooseString , %param2%, ComboBox2, %vwapWindowTitle%			// Series
	Control, ChooseString , %param3%, ComboBox3, %vwapWindowTitle%			// Symbol
	Control, Choose		  , %param4%, ComboBox4, %vwapWindowTitle%			// Expiry Date - Set by Position
	Control, ChooseString , %param5%, ComboBox5, %vwapWindowTitle%			// Opt Type
	Control, ChooseString , %param6%, ComboBox6, %vwapWindowTitle%			// Strike Price	
	
	ControlSetText, Edit1, 1,       %vwapWindowTitle%						// Set Interval as 1		
	ControlSend,    Edit1, {Enter}, %vwapWindowTitle%						// Request Data		
}

// Columns Expected Order - Start time, O, H, L, C, V
writeVwapData(){
	global vwapWindowTitle, VWAPBackfillFileName, DTBackfillFileName, scripAlias, VWAPSleepTime
	
	// Wait till all data is available
	// Just hope that sleep time is enough to fetch all data
	// When slow, NOW seems to fetch them in batches without necessarily a clear order. sort messes it up more
	// Better option - Can sort and try to verify that each minute from current time till 09:15 has been fetched 		
		
	Sleep, %VWAPSleepTime%
	
	ControlGet, vwapStats, List, , SysListView321, %vwapWindowTitle%		// Copy Data into vwapStats
	
	IfExist, %DTBackfillFileName%											// Clear Out DT data
	{
		file := FileOpen(DTBackfillFileName, "w" )	  
		if IsObject(file){
			file.Write(" ")
			file.Close()
		}
	}
	
	IfExist, %VWAPBackfillFileName%
	{				
		file := FileOpen(VWAPBackfillFileName, "w" )	  				    // := does not need %% for var
		if !IsObject(file){
			MsgBox "Can't open VWAP file for writing"
			ExitApp
		}
		
		AliasText := "name=" . scripAlias . "`n"
		file.Write(AliasText)												// Append AB Scrip Name		
		file.Write(vwapStats)												// Add Data
		file.Close()		
	}
	else{
		MsgBox "VWAP backfill file Not Found" 
		ExitApp
	}
}
 

josh1

Well-Known Member
Ok. Backfill can be done from VWAP/Hourly Statistics if NOW or Nest is open.Currently it is done in background after selecting scrips but yes it can be done by selecting all scrips from settings and the scrips need not be present in Marketwatch. However, possible with data table only if the scrip is present there. Can be done by tweaking existing code. Will see if I get time.
 

TracerBullet

Well-Known Member
Ok. Backfill can be done from VWAP/Hourly Statistics if NOW or Nest is open.Currently it is done in background after selecting scrips but yes it can be done by selecting all scrips from settings and the scrips need not be present in Marketwatch. However, possible with data table only if the scrip is present there. Can be done by tweaking existing code. Will see if I get time.
i haven't tested RTD recently, but earlier with backfill i had to
1) Switch from AB and Keep Nest at top
2) select time options and select scrips and then start and then wait for it to finish

So, What i meant was this.
1) No need to keep Nest at top.
2) Select scrips once and save their ids using GUI. Then have some input args option to backfill everything in list without seeing GUI or selecting anything. With 1 min bars its even easier and is direct replacement so you can update all bars with volume.

This then becomes your regular workflow - 1 click to run exe (with input args using batch file / shortcut) with Amibroker still in front. Even In NOW, the windows are minimized.
We can even include this in RTDMan to make automatic backfills, but i think its best to keep it seperate and avoid any possible interruption when ordering.

Again it depends on what you want. With above script i am able to backfill Nifty Future with Datatable reliably (so far) in couple of seconds while still watching AB.
 

josh1

Well-Known Member
i haven't tested RTD recently, but earlier with backfill i had to
1) Switch from AB and Keep Nest at top
2) select time options and select scrips and then start and then wait for it to finish

So, What i meant was this.
1) No need to keep Nest at top.
2) Select scrips once and save their ids using GUI. Then have some input args option to backfill everything in list without seeing GUI or selecting anything. With 1 min bars its even easier and is direct replacement so you can update all bars with volume.

This then becomes your regular workflow - 1 click to run exe (with input args using batch file / shortcut) with Amibroker still in front. Even In NOW, the windows are minimized.
We can even include this in RTDMan to make automatic backfills, but i think its best to keep it seperate and avoid any possible interruption when ordering.

Again it depends on what you want. With above script i am able to backfill Nifty Future with Datatable reliably (so far) in couple of seconds while still watching AB.
RTD is doing backfill in the background reliably with Amibroker in the front.There is no need to keep NOW/Nest at Top.
 

Similar threads