Your code works initially. Run the snippet.
It looks like using the .not() checks the DOM and not the current markup.
The first alert, contains your code, which works because they (DOM and HTML) are rendered the same. The DOM might not be equal to the HTML displayed in our browser after some events.
When you choose an option, I used 2 more alerts; one that uses .not() and another that uses jquery selector.
Notice that when you click an option, the HTML isn't modified. Your option fields with select attributes aren't changed, they are still as is. But in the DOM, the selected attributes are removed and only the ones clicked are actually 'selected'.
The last alert contains your expected values.
$(document).ready(function() {
// initial
alert("Initial; There are " + $("#mydd option").not(":selected").length + " not selected options.");
// when an option is clicked, DOM itself would unselect the other options, but it doesn't modify the markup
$("#mydd").click(function() {
alert("Using .not; There are " + $("#mydd option").not(":selected").length + " not selected options.");
// check if the 'selected' attribute exist
alert("Using jquery selector; There are " + $("#mydd option:not([selected]").length + " not selected options.");
});
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<select id="mydd" multiple searchable="Search here..">
<option value="" disabled selected>Choose your country</option>
<option selected="selected" value="1">USA</option>
<option selected="selected" value="2">Germany</option>
<option selected="selected" value="3">France</option>
<option value="4">Poland</option>
<option selected="selected" value="5">Japan</option>
<option value="6">Korea</option>
<option selected="selected" value="7">India</option>
</select>
