My first question is... Are you being represented by a buyer's agent? That is the first person you should go to for advice. If not, then you should keep in mind that you are dealing directly with the agent who ( when the home was listed ) was representing the seller. Now if he is representing you as well, you would have signed a "Consent to act" form which gives that agent the authorization to act for both parties otherwise known as "Duel Agency". Some people and many states consider this a "Conflict of Interest" as the agent in question has to look out for the interest of both parties. Keeping this in mind you need to be aware that as long as that agent is representing you, it is his/her fiduciary duty to guide and advise you as best as possible to protect your interests.
Long story short,... you can not be forced into paying closing costs if it was not agreed upon in the initial contract. Based on your agents advice,.. you need to make the choice that works best for you and your agent needs to move forward with your decision.
If this deal doesn't work out for you, I would strongly suggest using the services of a buyer's agent on the next transaction.
If I can be of service to you in the future.. I'm just a phone call away..!
I hope this helps... good luck to you...!!