How to await a coroutine with an event loop

ghz 2days ago ⋅ 4 views

I have a coroutine that has an event loop. How should I await that coroutine? I am using Python 3.7.3.

# The coroutine
async def main():
    #
    # This code written using apscheduler.schedulers.asyncio.AsyncIOScheduler
    # which has its *event loop* to run scheduled tasks
    # 
    # However, this code will await another coroutine, e.g.,
    # redis = await aioredis.create_redis('/tmp/redis.sock')

    try:
        asyncio.get_event_loop().run_forever()
    except:
        pass


if __name__ == '__main__':
    asyncio.run(main())

The code only runs the main event loop created by asyncio.run() but not the one by managed by apscheduler.

Update 1

However, if change the code to

# The coroutine
async def main():
    #
    # This code written using apscheduler.schedulers.asyncio.AsyncIOScheduler
    # which has its *event loop* to run scheduled tasks
    # 
    # However, this code will await another coroutine, e.g.,
    # redis = await aioredis.create_redis('/tmp/redis.sock')

    #try:
    #    asyncio.get_event_loop().run_forever()
    #except:
    #    pass


if __name__ == '__main__':
    asyncio.run(main())

The code runs main() once as the event loop managed by apscheduler does not run.

However, if change the code to

# The coroutine
async def main():
    #
    # This code written using apscheduler.schedulers.asyncio.AsyncIOScheduler
    # which has its *event loop* to run scheduled tasks
    # 
    # However, this code will await another coroutine, e.g.,
    # redis = await aioredis.create_redis('/tmp/redis.sock')

    try:
        asyncio.get_event_loop().run_forever()
    except:
        pass


if __name__ == '__main__':
    main()

The complains that the coroutine 'main' was never awaited.

Answer

In your code, you're encountering issues because you're trying to manage multiple event loops—one from asyncio.run() and another from apscheduler.schedulers.asyncio.AsyncIOScheduler—which is leading to confusion and incorrect behavior.

Let's break down what’s happening and how to fix it:

Problem Breakdown:

  1. Event Loop Management:

    • asyncio.run() creates a new event loop and runs the coroutine, then closes the loop when the coroutine finishes. This prevents any other event loop from being started in that process, including the one used by apscheduler.
    • You're attempting to use apscheduler's event loop, but calling asyncio.run() prevents it from working properly.

  2. Your attempts:

    • When you try to manually call asyncio.get_event_loop().run_forever(), you encounter issues because asyncio.run() is already managing the event loop.
    • The RuntimeWarning about "the coroutine 'main' was never awaited" occurs when you try to invoke the main() coroutine directly without await.

Solution:

You need to handle the event loop in a way that ensures both asyncio and apscheduler can work together. Here's how you can modify your code to properly integrate the two event loops:

Correct Code:

import asyncio
from apscheduler.schedulers.asyncio import AsyncIOScheduler

async def main():
    # Simulate an async task (e.g., connecting to Redis)
    print("Main coroutine started")
    await asyncio.sleep(1)
    print("Main coroutine completed")

    # Start APScheduler's event loop to run scheduled tasks
    scheduler = AsyncIOScheduler()
    scheduler.start()

    # Example of scheduling a job
    scheduler.add_job(print, 'interval', seconds=2, args=["Scheduled task"])
    
    # Keep the event loop running to allow APScheduler tasks to execute
    while True:
        await asyncio.sleep(1)

if __name__ == '__main__':
    # Running the main coroutine within the asyncio event loop
    asyncio.run(main())

Explanation of Changes:

  1. Combining APScheduler with AsyncIO: In this example, I use AsyncIOScheduler from apscheduler to schedule periodic tasks. The scheduler.start() method is called to start the scheduler's event loop.
  2. Keeping the Loop Running: Since asyncio.run() will run the main() coroutine and then stop the event loop, the while True loop with await asyncio.sleep(1) ensures that the event loop keeps running indefinitely to handle scheduled tasks.
  3. No Need for asyncio.get_event_loop().run_forever(): With this approach, asyncio.run() is used correctly, and there’s no need to manually call run_forever().

Key Points:

  • asyncio.run(main()) should be the entry point for running your coroutine, and it manages the event loop for you.
  • Use apscheduler.schedulers.asyncio.AsyncIOScheduler to integrate APScheduler with asyncio. Ensure that you start the scheduler with scheduler.start().
  • Inside the main() coroutine, after starting the scheduler, keep the event loop alive (with a while True loop or similar) so that APScheduler can continue to execute scheduled tasks.

This way, both your asyncio event loop and apscheduler's event loop will function properly together, allowing for scheduled tasks while still awaiting your coroutines.

#py