Javascript: Toggle div visibility in two steps

This question maybe stupid for many here. I have a bunch of divs and want to make them appear/disappear on click with special behaviour:

On-load state: all divs visible

1. click: all divs disappear, except for the one that was selected when clicking

2. to n-th click: toggle visibility for the div that was selected when clicking

What I've got so far:

                function toggle_visibility(id) {
            var e = document.getElementById(id);
            if(e.style.display == 'block')
              e.style.display = 'none';
            else
              e.style.display = 'block';
        }

                function toggle_class(id) {
            var thisElem = document.getElementById(id);
            var invisible = "invisible";
            var visible = "visible";
            var classes = thisElem.classList;
            if (classes == invisible) {
                thisElem.className = thisElem.className.replace(invisible, visible);
            }
            else {
                thisElem.className = thisElem.className.replace(visible, invisible);
            }
        }

<ul id='list'>
        <li id='1-i' class='visible'><a href='javascript:return false;' onclick="toggle_visibility('1'); toggle_class('1-i');">Toggle DIV #1</a></li>
        <li id='2-i' class='visible'><a href='javascript:return false;' onclick="toggle_visibility('2'); toggle_class('2-i');">Toggle DIV #2</a></li>
        <li id='3-i' class='visible'><a href='javascript:return false;' onclick="toggle_visibility('3'); toggle_class('3-i');">Toggle DIV #3</a></li>
        <li id='4-i' class='visible'><a href='javascript:return false;' onclick="toggle_visibility('4'); toggle_class('4-i');">Toggle DIV #4</a></li>
</ul>


<div id='1' style='display: block;'><h3>DIV #1</h3></div>
<div id='2' style='display: block;'><h3>DIV #2</h3></div>
<div id='3' style='display: block;'><h3>DIV #3</h3></div>
<div id='4' style='display: block;'><h3>DIV #4</h3></div>

This code shows all divs on page-load, then toggles visibility for the selected div on click. So on first click only the selected div will disappear with all others staying still visible - the opposite of what I want. Though from second click on, this behaviour is the desired one.

I have found similar other threads (like this one), but their issues seem to add complexity I'd like to avoid.

Thanks a lot for your help!

---

Edit: Now I tried to update function toggle_class(id) { following Arun P Johny's example:

        var firstrun = true;

        function toggle_class(id) {
            var thisElem = document.getElementById(id);
            var invisible = "invisible";
            var visible = "visible";
            if (thisElem.className == 'invisible' && !firstrun) {
                thisElem.className = thisElem.className.replace(invisible, visible);
            } else {
                thisElem.className = thisElem.className.replace(visible, invisible);
            }
            if (firstrun) {
                var children = document.getElementsByClassName('visible');
                for (var i = 0; i < children.length; i++) {
                    if (children[i].id != id) {
                        children[i].className = thisElem.className.replace(visible, invisible);
                    }
                }
            }
            firstrun = false;
        }

The result is somewhat confusing: On first click, the selected element changes its class to invisible (which I do understand, since the script tries to replace the class visible with invisible for all elements). So this is not the behaviour I want, the clicked element is supposed to keep the class visible (until it is clicked again, since this is when the div disappears).

And the even more confusing part to me: Not all other elements change their class to invisible, but only every second element.

What did I do wrong?