return documents based on orders

How does a system identifies the correct inventory cost of a sales return is created based on a sales order and a goods receipt document (based on the sales order) has a cost that's different from the cost in the sales order?

Actually, in the next release we'll disable creating returns based on order document. You'll only be able to create them based on warehouse documents (goods receipt and shipment) and invoices. In this case the cost will be taken from the warehouse documents / invoices.