When I use httplib for my OAUTH in Python, I always get "CannotS

ghz 2days ago ⋅ 11 views

When I use httplib for my OAUTH in Python, I always get "CannotSendRequest" and then "

Traceback:
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/base.py" in get_response
  92.                 response = callback(request, *callback_args, **callback_kwargs)
File "/home/ea/ea/hell/life/views.py" in linkedin_auth
  274.     token = oauth_linkedin.get_unauthorised_request_token()
File "/home/ea/ea/hell/life/oauth_linkedin.py" in get_unauthorised_request_token
  52.     resp = fetch_response(oauth_request, connection)
File "/home/ea/ea/hell/life/oauth_linkedin.py" in fetch_response
  42.         connection.request(oauth_request.http_method,url)
File "/usr/lib/python2.6/httplib.py" in request
  874.             self._send_request(method, url, body, headers)
File "/usr/lib/python2.6/httplib.py" in _send_request
  891.         self.putrequest(method, url, **skips)
File "/usr/lib/python2.6/httplib.py" in putrequest
  778.             raise CannotSendRequest()

Exception Type: CannotSendRequest at /linkedin/auth
Exception Value:

And then, sometimes I get: BadStatusLine error instead of this.

It's pretty random. I don't know when or why they happen. It happens more frequently when I'm running the Django development server (and less frequently when in APACHE2...but it still happens at random times). When this error happens, I have to restart my server.

Answer

The error CannotSendRequest and BadStatusLine from httplib often indicate an issue with the HTTP connection. These errors typically arise from:

  1. Persistent Connections Issues: The connection object (connection) may be reused when it's in an invalid or closed state.
  2. Concurrency Problems: If the connection object is shared across multiple threads or requests without proper locking or management.
  3. Network Instability: Temporary network issues causing the connection to drop unexpectedly.
  4. Bug in httplib or the HTTP Server: Particularly with older Python versions (e.g., Python 2.6).

Since you're using Python 2.6 (a very old and unsupported version), the likelihood of encountering bugs in the HTTP library is higher.

Steps to Mitigate the Problem

  1. Ensure a Fresh Connection for Each Request Avoid reusing the same HTTP connection object for multiple requests. Create a new connection object for each HTTP request.

    Update your fetch_response function to explicitly close the connection after use:

    import httplib

def fetch_response(oauth_request, connection):
    try:
        connection.request(oauth_request.http_method, oauth_request.to_url())
        response = connection.getresponse()
        return response.read()
    finally:
        # Ensure the connection is closed
        connection.close()

  2. Use a Modern Library Instead of relying on httplib, use a more modern and robust HTTP library such as requests. This library handles connection pooling, retries, and other complexities internally.

    Example:

    import requests

def fetch_response(oauth_request):
    response = requests.request(
        method=oauth_request.http_method,
        url=oauth_request.to_url()
    )
    response.raise_for_status()  # Raise an error for HTTP errors
    return response.text

  3. Check for Thread Safety If your application uses multithreading (e.g., Django development server or Apache with WSGI), ensure that the connection object is not shared across threads. Sharing connections without synchronization can cause unpredictable behavior.

  4. Upgrade Your Python Version Python 2.6 is outdated and no longer supported. Consider upgrading to Python 3.x to benefit from more stable and well-maintained libraries.

  5. Handle BadStatusLine Gracefully Implement retry logic to handle temporary connection issues like BadStatusLine. Wrap your HTTP call in a retry loop:

    import time
from httplib import BadStatusLine

def fetch_response(oauth_request, connection, retries=3):
    for attempt in range(retries):
        try:
            connection.request(oauth_request.http_method, oauth_request.to_url())
            response = connection.getresponse()
            return response.read()
        except BadStatusLine:
            if attempt < retries - 1:
                time.sleep(1)  # Small delay before retrying
            else:
                raise
        finally:
            connection.close()

  6. Debugging Tips

    • Log the connection state before making the request to understand why it fails.
    • Add logging for each request to identify patterns leading to the error.
    • Test with a newer Django version on Python 3.x if possible.

  7. Development vs Production

    • The Django development server is single-threaded and not optimized for concurrent connections. Use a proper WSGI server like Gunicorn or uWSGI in production to minimize such issues.

By following these steps, you can mitigate or eliminate the random CannotSendRequest and BadStatusLine errors.

#py