Your memory is not the problem. Your memory is appearing to fail you because the way you are using it is not suitable for learning languages. There are some of you who have trouble with memorising words (etc). Then there are some of you who might have a real good "memory" and can remember the words ( etc ) you are learning but then, you don't use them when you need them. Sound familiar?
I will give you a number of scenarios here for you to consider. Read them and see what you think.
1. What infant ever memorises new words in the way we may be use to?
2. Would you ever have to memorise something that a close friend or family member told you about another close friend or family member?
3. When you read a newspaper article or book, watch a movie, read an interesting post on social media, do you set about memorising it so you can tell someone else about it?
If you answered 2 & 3 in the way that 99.9% of people would answer it, you will recognise that your memory is just fine. Put together with question 1, you may understand ( maybe relunctantly! 🙂 ) that our memory can work fine without memorisation. Let's call this way, for now, natural learning (A).
The vast majority of people who have learned languages formally would have been taught in ways that emphasise memorisation (B) not the other way, A. When people learn languages, informally, that is "on the street" then they are learning the natural way. Seldom do these people spend time on memorisation.
In fact unless a person moves from A to B at some stage, they are destined to only get to a certain level. Without learning to improve as you are using the language, your level will hit a plateau from which it will struggle to improve. This happens because the learner keeps looking for help ( in the past from the teacher or a book), rather than recognising that improvement can happen through better noticing how s/he and others use English.
It is possible to "formally" learn languages in ways that emphasise what I would call "natural learning"