Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference

Question

Given the following examples, why is outerScopeVar undefined in all cases?

var outerScopeVar;

var img = document.createElement('img');
img.onload = function() {
    outerScopeVar = this.width;
};
img.src = 'lolcat.png';
alert(outerScopeVar);



var outerScopeVar;
setTimeout(function() {
    outerScopeVar = 'Hello Asynchronous World!';
}, 0);
alert(outerScopeVar);



// Example using some jQuery
var outerScopeVar;
$.post('loldog', function(response) {
    outerScopeVar = response;
});
alert(outerScopeVar);



// Node.js example
var outerScopeVar;
fs.readFile('./catdog.html', function(err, data) {
    outerScopeVar = data;
});
console.log(outerScopeVar);



// with promises
var outerScopeVar;
myPromise.then(function (response) {
    outerScopeVar = response;
});
console.log(outerScopeVar);



// with observables
var outerScopeVar;
myObservable.subscribe(function (value) {
    outerScopeVar = value;
});
console.log(outerScopeVar);



// geolocation API
var outerScopeVar;
navigator.geolocation.getCurrentPosition(function (pos) {
    outerScopeVar = pos;
});
console.log(outerScopeVar);

Why does it output undefined in all of these examples? I don't want workarounds, I want to know why this is happening.

Note: This is a canonical question for JavaScript asynchronicity. Feel free to improve this question and add more simplified examples which the community can identify with.

Answer

One word answer: asynchronicity.

Forewords

This topic has been iterated at least a couple of thousands of times here in Stack Overflow. Hence, first off I'd like to point out some extremely useful resources:

@Felix Kling's answer to "How do I return the response from an asynchronous call?". See his excellent answer explaining synchronous and asynchronous flows, as well as the "Restructure code" section.
@Benjamin Gruenbaum has also put a lot of effort into explaining asynchronicity in the same thread.
@Matt Esch's answer to "Get data from fs.readFile" also explains asynchronicity extremely well in a simple manner.

The answer to the question at hand

Let's trace the common behavior first. In all examples, the outerScopeVar is modified inside of a function. That function is clearly not executed immediately; it is being assigned or passed as an argument. That is what we call a callback.

Now the question is, when is that callback called?

It depends on the case. Let's try to trace some common behavior again:

img.onload may be called sometime in the future when (and if) the image has successfully loaded.
setTimeout may be called sometime in the future after the delay has expired and the timeout hasn't been canceled by clearTimeout. Note: even when using 0 as delay, all browsers have a minimum timeout delay cap (specified to be 4ms in the HTML5 spec).
jQuery $.post's callback may be called sometime in the future when (and if) the Ajax request has been completed successfully.
Node.js's fs.readFile may be called sometime in the future when the file has been read successfully or thrown an error.

In all cases, we have a callback that may run sometime in the future. This "sometime in the future" is what we refer to as asynchronous flow.

Asynchronous execution is pushed out of the synchronous flow. That is, the asynchronous code will never execute while the synchronous code stack is executing. This is the meaning of JavaScript being single-threaded.

More specifically, when the JS engine is idle -- not executing a stack of (a)synchronous code -- it will poll for events that may have triggered asynchronous callbacks (e.g. expired timeout, received network response) and execute them one after another. This is regarded as [Event Loop](https://developer.mozilla.org/en- US/docs/Web/JavaScript/Guide/EventLoop).

That is, the asynchronous code highlighted in the hand-drawn red shapes may execute only after all the remaining synchronous code in their respective code blocks have executed:

async code highlighted

In short, the callback functions are created synchronously but executed asynchronously. You can't rely on the execution of an asynchronous function until you know it has been executed, and how to do that?

It is simple, really. The logic that depends on the asynchronous function execution should be started/called from inside this asynchronous function. For example, moving the alerts and console.logs inside the callback function would output the expected result because the result is available at that point.

Implementing your own callback logic

Often you need to do more things with the result from an asynchronous function or do different things with the result depending on where the asynchronous function has been called. Let's tackle a bit more complex example:

var outerScopeVar;
helloCatAsync();
alert(outerScopeVar);

function helloCatAsync() {
    setTimeout(function() {
        outerScopeVar = 'Nya';
    }, Math.random() * 2000);
}

Note: I'm using setTimeout with a random delay as a generic asynchronous function; the same example applies to Ajax, readFile, onload, and any other asynchronous flow.

This example clearly suffers from the same issue as the other examples; it is not waiting until the asynchronous function executes.

Let's tackle it by implementing a callback system of our own. First off, we get rid of that ugly outerScopeVar which is completely useless in this case. Then we add a parameter that accepts a function argument, our callback. When the asynchronous operation finishes, we call this callback, passing the result. The implementation (please read the comments in order):

// 1. Call helloCatAsync passing a callback function,
//    which will be called receiving the result from the async operation
helloCatAsync(function(result) {
    // 5. Received the result from the async function,
    //    now do whatever you want with it:
    alert(result);
});

// 2. The "callback" parameter is a reference to the function which
//    was passed as an argument from the helloCatAsync call
function helloCatAsync(callback) {
    // 3. Start async operation:
    setTimeout(function() {
        // 4. Finished async operation,
        //    call the callback, passing the result as an argument
        callback('Nya');
    }, Math.random() * 2000);
}

Code snippet of the above example:

// 1. Call helloCatAsync passing a callback function,
//    which will be called receiving the result from the async operation
console.log("1. function called...")
helloCatAsync(function(result) {
    // 5. Received the result from the async function,
    //    now do whatever you want with it:
    console.log("5. result is: ", result);
});

// 2. The "callback" parameter is a reference to the function which
//    was passed as an argument from the helloCatAsync call
function helloCatAsync(callback) {
    console.log("2. callback here is the function passed as argument above...")
    // 3. Start async operation:
    setTimeout(function() {
    console.log("3. start async operation...")
    console.log("4. finished async operation, calling the callback, passing the result...")
        // 4. Finished async operation,
        //    call the callback passing the result as argument
        callback('Nya');
    }, Math.random() * 2000);
}

Most often in real use cases, the DOM API and most libraries already provide the callback functionality (the helloCatAsync implementation in this demonstrative example). You only need to pass the callback function and understand that it will execute out of the synchronous flow and restructure your code to accommodate for that.

You will also notice that due to the asynchronous nature, it is impossible to return a value from an asynchronous flow back to the synchronous flow where the callback was defined, as the asynchronous callbacks are executed long after the synchronous code has already finished executing.

Instead of returning a value from an asynchronous callback, you will have to make use of the callback pattern, or... Promises.

Promises

Although there are ways to keep the callback hell at bay with vanilla JS, promises are growing in popularity and are currently being standardized in ES6 (see [Promise - MDN](https://developer.mozilla.org/en- US/docs/Web/JavaScript/Reference/Global_Objects/Promise)).

Promises (a.k.a. Futures) provide a more linear, and thus pleasant, reading of the asynchronous code, but explaining their entire functionality is out of the scope of this question. Instead, I'll leave these excellent resources for the interested: