In this post I will try to cover all steps, that I had to take to fix this issue.
PS.
I am aware, that there is a section dedicated to solving it at Docker official resource https://docs.docker.com/docker-for-windows/#firewall-rules-for-shared-drives
It did not help me.

Little intro regarding setup. I am using Windows 10 and Hyper-V

Steps:
1. Disable Firewall
2. Disable Anti-Virus
3. Share Drive, that you are trying to allow in Docker with user "Everyone"
4. Go to Network and Sharing Center > Change adapter settings > Select one that Docker VM uses and uncheck "File and Printer Sharing"
5. Go to Network and Sharing Center > Change adapter settings > Select one that Docker VM uses and check "File and Printer Sharing"
6. Restart Docker
7. Hopefully Profit