I would like to plant also basil, thyme and dill there, but would like that the snake goes away. Any suggestions?

Sounds like a gater snake. They chomp on you but not poisonous. Best way to tell a poisonous snake is if its head is triangular shaped its poisonous, except for a coral snake I think.

If it's a thin, yard long or so black snake with thin yellow/white stripes from nose to tail (not horizontal bands) it's a variety of garden snake. They're really helpful keeping out mice and small critters from your garden, and are usually very shy. They like sunny, rocky places or railroad ties.

What you most probably have is a garter snake. Not knowing where you live makes it a bit more difficult, but I am not aware of a snake with a yellow stripe being poisonous. In North America the poisonous snakes are: rattle snake, copperhead, coral snake and water moccasin. None or black with a yellow stripe though the moccasin is closest.

Another clue is if it has a triangular head. If it does, chances are it is one of the above and poisonous. Otherwise, it is probably very helpful to have. Most snakes are NOT poisonous and help with insects and vermin.

Here is a website among many that may help you identify your snake. And, assure you it is not poisonous. Good luck.
http://www.pitt.edu/~mcs2/herp/SoNA.html
http://www.montgomerycountymd.gov/deptmpl.asp?url=/content/dep/herps/snakes.asp

You are describing a garden snake. It is not poisonous and is actually very good for your garden. This snake feeds on small animals such as mice, that come into your garden to feed on the herb's there. So-named the "Garden Snake", because he/she takes care of your garden!
I would leave it there, especially if you value the health of the herb's in your garden!
also it is a non-aggressive snake which mean's you can pick it up and hold it, and even let it wrap around your hands, wrists, arms, whatever, just make sure you put it back in your garden so that it can do it's job of protecting your herb's!