honestly do not understand how memory affects performance
It does not directly. Its just RAM sets a limit on TC that can be set with --lookup-gap.
For example with 4gb ram I can run two 280x with your settings and get 130khash. But only two of them, third one wont budge till i increase gap to three. But result of raising gap iss always reduced hash with the same TC.