Turn of Silence Supression. If you have already done that then I think you are having the usual Xlite - Asterisk experience. At least I have the same problems with it.