I found the answer here:

Let’s say that you have a submodule in your project called ’submodule1? and it’s in the following path: ‘vendors/submodule1?. In git there are 3 traces of this this submodule:
1) .gitmodules
2) .git/config
3) the submodule entry in the index/commit itself.

To remove the first two, is really simple, you just edit those files and remove the lines that specify the submdoule. In order to delete the third and last trace of the submodule in git, you need to type the following command:
git rm --cached path/to/submodule
Note: Do not put a trailing slash at the end of path. If you put a trailing slash at the end of the command, it will fail.

In the example above, we would type (do not include trailing slash after submodule1):
git rm --cached vendors/submodule1

I’ll put here, so I hope to find it easily later on.