Finance Programming is Challenging
As some may know, I’ve been working with my brother for the last two years on financial algorithms to perform arbitrage trades using what little knowledge we had of stable assets and volatile assets. This was essentially a means to do foreign exchange without having to use traditional financial instruments that were prohibitively expensive.
We were able to succeed in proving the concept from a technical perspective but not from a scaling perspective. Ultimately, what we realized was that the algorithms and compute overhead to do the data processing was beyond our reach with our current software architecture. We started this year investigating ways to improve performance of our data processing without major capital investments in computing power.
Scaling
We’re currently kicking a can down the road that we shouldn’t be. We need to switch the architecture to something that can be spread horizontally across small compute but also we need to change something that scales vertically to utilise all resources on a computer.
Currently, we have workers that are largely dictated arbitrarily by our settings file. We have multiple classes of workers but some workers are only useful for one thing and having multiple of them doesn’t split up the work in any meaningful way. This is a critical problem with our architecture.
The reason why it was setup this way was because of the memory sharing constraints of NodeJS meaning that there was excessive serialization and data syncing issues that results in a lot of overhead. Moving all of the data over to the database introduces extreme latency especially with so many workers talking. These are all solved problems but for various reasons we haven’t pursued them.
The main reason is laziness.
Time Capital vs Motivation Capital
Time capital is great but its useless without motivation capital. I think the mental blocks and barriers I’ve encountered in the last few months of largely working alone on the major parts of the project have taken a larger toll than I recognized. I’m tired of learning without actually making money. I’m tired of studying software concepts, applying them, only to realize it wasn’t enough. In the last 2 years, I’ve learned a lot about the computer programming world. I would love to be able to continue programming in some fashion but without stable income, I’ll likely have to return to the previous semiconductor world in largely the same capacity.
I have weird feelings. I want to be competent enough to make this work but I also want actual progress along the way. I’ve done coding sprints and hackathons only to have none of them actually result in financial output. Actually, some of them resulted in negative results. Learning for the sake of learning is great but it’s impossible to do with a looming debt over your head. A debt that didn’t even need to happen.
Motivation is what makes time capital work. I have a surplus of the latter without any of the former. This is the most frustrating place I’ve been in for a long time. Probably since I graduated university without employment.
I could rant more… but I won’t.
Signing off until I have more to work through.