The eye of the needle was a famous gate in Jerusalem.  It was so named because it was very small.  A rich man going to heaven being like a camel going through the eye of the needle was particularly apt to his contemporary audience.  Not only was it very difficult, but you'd have to remove the packs and possessions, then the camel would have to bow down and be on its knees.  The lesson was one of humility, not of impossibility.