It depends on what you're going, but you should go to prof. in math optimization in your university in which you're right now or which you graduated and you should ask him directly.
I implemented solvers for several problems reduced to convex optimization (http://cs229.stanford.edu/proj2017/) - cvx4ml which works faster then similar solution from SkLearn, and I passed 24 hour exam to Stephen Boyd, so I can give advice what you can do and describe for your very rough plan:
So you're going to create own package I will write step-by-step instruction:
- You should create library for work with dense matrix/vectors
- You should create library for work with sparse matrix/vectors
- Implement around 20 different algorithms to solve system of linear equations (because depend on situation you will need different of them)
- Introduce concepts how describe constraints, function domain, etc in your programming language or you system that you created.
- Implement several norms and dual norms evaluation, some factorization technics like LU, Cholesky.
Implement custom simple conic solver for non-negative orthont cone. And it depends on what you're going todo.
6.a - write solver based on interior point method.
6.b - write solver with support distributed optimization
6.c - write solver based on some projected subgradient method.
Improve it to support other cones
- Enhanced you solver with step "5"
And if you want to be at level of CVXPY then
- Implement parsing of program description like CVXPY do and transform problem into conic form.
p.s. If you feel sloppy with some of this topics, then:
Read linear algebra book which wrote prof. from your university
Look in the youtube into EE263 with S.Boyd, EE364A with S.Boyd, EE364B with S.Boyd.