I see you are using bigdecimal, why are you not using integer math? I would use primitive objects like longs to do all math operations.
There are advantages and disadvantages in using integer math. We decided that the performance and size benefits did not outweigh the extra complexities and potential bugs with using integer values. O(n) operations will still be O(n), regardless of the object types.
I don't see any trading engine or even an outline, I hope you are not using scala for the trading engine, that should be written in C or C++ to be anything considered commercial grade.
Trading engine is in the SQL files. The trading engine is written in PL/pgSQL. The lookups are done using a cursor on the orders table. The orders table is indexed using sorted multi-column partial indexes so the engine may read the orders from the index without doing any sorting. This was verified with EXPLAIN ANALYZE in Postgres.
I haven't really looked into security, but I would treat this as a hobby project not a commercial grade anything.
Some details on security:
https://github.com/txbits/txbits/wiki/SecurityCan anyone help me install this to my server?
Use the wiki for instructions on setting up a development environment. Wait for a stable release before deploying it to a server.