Question
I have written the same API application with the same function in both
FastAPI and Flask. However, when returning the JSON, the format of
data differs between the two frameworks. Both use the same json
library and
even the same exact code:
import json
from google.cloud import bigquery
bigquery_client = bigquery.Client()
@router.get('/report')
async def report(request: Request):
response = get_clicks_impression(bigquery_client, source_id)
return response
def get_user(client, source_id):
try:
query = """ SELECT * FROM ....."""
job_config = bigquery.QueryJobConfig(
query_parameters=[
bigquery.ScalarQueryParameter("source_id", "STRING", source_id),
]
)
query_job = client.query(query, job_config=job_config) # Wait for the job to complete.
result = []
for row in query_job:
result.append(dict(row))
json_obj = json.dumps(result, indent=4, sort_keys=True, default=str)
except Exception as e:
return str(e)
return json_obj
The returned data in Flask was dict:
{
"User": "fasdf",
"date": "2022-09-21",
"count": 205
},
{
"User": "abd",
"date": "2022-09-27",
"count": 100
}
]
While in FastAPI was string:
"[\n {\n \"User\": \"aaa\",\n \"date\": \"2022-09-26\",\n \"count\": 840,\n]"
The reason I use json.dumps()
is that date
cannot be itterable.
Answer
The wrong approach
If you serialise the object before returning it, using json.dumps()
(as
shown in your example), for instance:
import json
@app.get('/user')
async def get_user():
return json.dumps(some_dict, indent=4, default=str)
the JSON object that is returned will end up being serialised twice , as FastAPI will automatically serialise the return value behind the scenes. Hence, the reason for the output string you ended up with:
"[\n {\n \"User\": \"aaa\",\n \"date\": \"2022-09-26\",\n ...
Solutions
Have a look at the available solutions, as well as the explanation given below as to how FastAPI/Starlette works under the hood.
Option 1
The first option is to return data (such as dict
, list
, etc.) as usual—
i.e., using, for example, return some_dict
—and FastAPI, behind the scenes,
will automatically convert that return value into
JSON, after first
converting the data into JSON-compatible data, using the
jsonable_encoder
. The
jsonable_encoder
ensures that objects that are not serializable, such as
datetime
objects, are
converted to a str
. Then, FastAPI will put that JSON-compatible data inside
of a [JSONResponse
](https://fastapi.tiangolo.com/advanced/custom-
response/?h=jsonresp#jsonresponse), which will return an application/json
encoded response to the client (this is also explained in Option 1 of this
answer). The JSONResponse
,
as can be seen in Starlette's source code
here,
will use the Python standard json.dumps()
to serialise the dict
(for
alternatvie/faster JSON encoders, see this
answer and this
answer).
Example
from datetime import date
d = [
{"User": "a", "date": date.today(), "count": 1},
{"User": "b", "date": date.today(), "count": 2},
]
@app.get('/')
def main():
return d
The above is equivalent to:
from fastapi.responses import JSONResponse
from fastapi.encoders import jsonable_encoder
@app.get('/')
def main():
return JSONResponse(content=jsonable_encoder(d))
Output:
[{"User":"a","date":"2022-10-21","count":1},{"User":"b","date":"2022-10-21","count":2}]
Returning a JSONResponse
or a custom Response
directly (it is demonstrated
in Option 2 below), as well as any other response class that inherits from
Response
(see FastAPI's documentation
here, as well as
Starlette's documentation here and
responses' implementation
here),
would also allow one to specify a custom status_code
, if they will. The
implementation of FastAPI/Starlette's JSONResponse
class can be found
here,
as well as a list of HTTP codes that one may use (instead of passing the [HTTP
response status code](https://developer.mozilla.org/en-
US/docs/Web/HTTP/Status) as an int
directly) can be seen
here.
Example:
from fastapi import status
from fastapi.responses import JSONResponse
from fastapi.encoders import jsonable_encoder
@app.get('/')
def main():
return JSONResponse(content=jsonable_encoder(d), status_code=status.HTTP_201_CREATED)
Option 2
If, for any reason (e.g., trying to force some custom JSON format), you have
to serialise the object before returning it, you can then [return a custom
Response
directly](https://fastapi.tiangolo.com/advanced/response-
directly/#returning-a-custom-response), as described in this
answer. As per the
[documentation](https://fastapi.tiangolo.com/advanced/response-
directly/#notes):
When you return a
Response
directly its data is not validated, converted (serialized), nor documented automatically.
Additionally, as described [here](https://fastapi.tiangolo.com/advanced/custom- response/?h=jsonresp#response):
FastAPI (actually Starlette) will automatically include a Content-Length header. It will also include a Content-Type header, based on the
media_type
and appending a charset for text types.
Hence, you can also set the media_type
to whatever type you are expecting
the data to be; in this case, that is application/json
. Example is given
below.
Note 1 : The JSON outputs posted in this answer (in both Options 1 & 2)
are the result of accessing the API endpoint through the browser directly
(i.e., by typing the URL in the address bar of the browser and then hitting
the enter key). If you tested the endpoint through Swagger UI at /docs
instead, you would see that the indentation differs (in both options). This is
due to how Swagger UI formats application/json
responses. If you needed to
force your custom indentation on Swagger UI as well, you could avoid
specifying the media_type
for the Response
in the example below. This
would result in displaying the content as text , as the Content-Type
header would be missing from the response, and hence, Swagger UI couldn't
recognise the type of the data, in order to custom-format them (in case of
application/json
responses).
Note 2 : Setting the default
argument to str
in
json.dumps()
is
what makes it possible to serialise the date
object, otherwise if it wasn't
set, you would get: TypeError: Object of type date is not JSON serializable
.
The default
is a function that gets called for objects that can't otherwise
be serialized. It should return a JSON-encodable version of the object. In
this case it is str
, meaning that every object that is not serializable, it
is converted to string. You could also use a custom function or JSONEncoder
subclass, as demosntrated
[here](https://stackoverflow.com/questions/11875770/how-to-overcome-datetime-
datetime-not-json-serializable), if you would like to serialise an object in a
custom way. Additionally, as mentioned in Option 1 earlier, one could instead
use alternative JSON encoders, such as orjson
, that might improve the
application's performance compared to the standard json
library (see this
answer and this
answer).
Note 3 : FastAPI/Starlette's
Response
accepts as a content
argument either a str
or bytes
object. As shown in
the implementation
here,
if you don't pass a bytes
object, Starlette will try to encode it using
content.encode(self.charset)
. Hence, if, for instance, you passed a dict
,
you would get: AttributeError: 'dict' object has no attribute 'encode'
. In
the example below, a JSON str
is passed, which will later be encoded into
bytes
(you could alternatively encode it yourself before passing it to the
Response
object).
Example
from fastapi import Response
from datetime import date
import json
d = [
{"User": "a", "date": date.today(), "count": 1},
{"User": "b", "date": date.today(), "count": 2},
]
@app.get('/')
def main():
json_str = json.dumps(d, indent=4, default=str)
return Response(content=json_str, media_type='application/json')
Output:
[
{
"User": "a",
"date": "2022-10-21",
"count": 1
},
{
"User": "b",
"date": "2022-10-21",
"count": 2
}
]