You might of noticed in the RMAH in beta the fee is being displayed as 15% for a item, and a 15% fee does come of the item when sold "I tested it".
Looks like blizzard whats a bigger cut of the sales "on items that sell for a decent amount"
It makes sense from them to do it this way as a % fee on the sale of a $100 item gives them far more profit than a flat fee.
There is no official word on this yet, but perhaps it will carry over to release.
Its pretty logical from a bussiness point of view, and for your average joe wanting to sell that nice drop he got for abit of side cash, the only way to do it. If they don't setup the fees to increase proportional with the trading, what is to stop some hedge fund coming in and throwing a couple hundred thousand at this?