At the end of the tenancy, the landlord can deduct from a deposit for unpaid rent or damage to the premises. The damage has to give rise to compensation. A landlord cannot deduct a deposit for ordinary wear and tear of the property which comes with occupation.
If the landlord fails to protect the deposit he/she will be in breach of the law. It is unlikely the landlord can deduct from the unprotected deposit where a check in report has also not been completed as there is no evidence to support the condition of the property at the start of the tenancy. It is always best practice for both the landlord and the tenant to take an accurate account of the property with descriptions/pictures at the start of the tenancy.
As a tenant you may request for it to be returned or apply to your local county court if you believe the landlord has not used a tenancy deposit scheme when they should have. You can use Form N208 to submit your claim alongside a fee of £308 (which you can claim back from the landlord if you win the case), if the court finds the landlord has not protected your deposit, it can order the landlord to repay it to you (or up to 3 times the amount of the deposit as compensation) or pay it into a TDS bank account within 14 days.
Please seek legal advice for further information.