how to save the output (json file)of a subprocess that is in a l

ghz 12hours ago ⋅ 7 views

how to save the output (json file)of a subprocess that is in a loop

this code send generates a random json file of user ids provided and btw range is also given..

so this code outputs 50 jsons for each user.

import faker
import json
from faker import Faker
import random
from random import randint
import subprocess
import json
import os
#subprocess.call([""])
from pprint import pprint

ids= ('5cda','6cda')

fake = Faker('en_US')

for ind in ids:
    cont = []
    #Overall dictionary with first and user_ids
    dct = {}
    for idx in range(50):

        sms =  {
            "id":"AB-asfgw",
            "body": fake.text(),
            "mime": fake.ean(),
            "hashed": fake.ean(),
            "pid": fake.ean(),
            "user_id": ind,
            "text": fake.sentence()
        }
        cont.append(sms)

    dct['messages'] = cont
    dct['user_id'] = ind
    #print(dct)
    f_name = '{}.json'.format(ind)
    with open(f_name, 'w') as fp:
        #Save the dictionary
        json.dump(dct, fp, indent=4)
        print('saved {}'.format(f_name))    

auth = "authorization: token 1324"
file = "5cda.json"
fd=open("5cda.json")
json_content = fd.read()
fd.close()


subprocess.run(["grpcurl", "-plaintext","-H", auth,"-d",json_content,"-format","json","100.20.20.1:5000","api.Service/Method"])

this loop.py code loops the first code 20 times

from datetime import datetime
import faker
import json
from faker import Faker
import random
from random import randint
import subprocess
import json
import os
#subprocess.call([""])
from pprint import pprint
import subprocess
import sys



for i in range(20):
    subprocess.call(['python','grploop1.py'])

i need to save the output of loop.py code for each loop. and store that json. example : we are looping the first code for 20 times in loop.py so ineed the output stored should be like 5cda1.json ........ 5cda20.json and 6cda1.json..... 6cda20.json

here we are giving two user ids ids= ('5cda','6cda') so output would be total 40 json files.

Answer

To achieve your goal of saving the output of each loop in loop.py and naming the resulting files in the format 5cda1.json, 5cda2.json, ..., 6cda1.json, 6cda20.json, we can modify your script slightly. Here's how we can do this:

Steps:

  1. Modify the grploop1.py (the original script) to accept a parameter indicating which iteration it is.
  2. Modify loop.py to pass the iteration number and save the output accordingly.

1. Modify grploop1.py to Accept the Iteration Number

In grploop1.py, we need to modify it to accept an iteration number as an argument and save the file with the iteration number included.

Here’s how to modify grploop1.py:

import faker
import json
from faker import Faker
import random
from random import randint
import subprocess
import json
import os
from pprint import pprint
import sys

# Fetch iteration number from arguments
iteration = sys.argv[1]  # For example, 1, 2, 3, ..., 20
ids = ('5cda', '6cda')

fake = Faker('en_US')

for ind in ids:
    cont = []
    # Overall dictionary with first and user_ids
    dct = {}
    for idx in range(50):

        sms =  {
            "id":"AB-asfgw",
            "body": fake.text(),
            "mime": fake.ean(),
            "hashed": fake.ean(),
            "pid": fake.ean(),
            "user_id": ind,
            "text": fake.sentence()
        }
        cont.append(sms)

    dct['messages'] = cont
    dct['user_id'] = ind
    # Print out the file name to be saved (with iteration number)
    f_name = '{}{}_{}.json'.format(ind, iteration, idx + 1)  # Add iteration and message index to file name
    with open(f_name, 'w') as fp:
        # Save the dictionary
        json.dump(dct, fp, indent=4)
        print(f'Saved {f_name}')

    # Optionally send the generated file over grpcurl if needed
    # auth = "authorization: token 1324"
    # subprocess.run(["grpcurl", "-plaintext", "-H", auth, "-d", json_content, "-format", "json", "100.20.20.1:5000", "api.Service/Method"])

Key Changes:

  • The script now takes an argument iteration which specifies the current iteration number (1, 2, ..., 20).
  • The output filenames are now in the format user_id_iteration_message_num.json. For example, 5cda1_1.json, 5cda1_2.json, ..., 6cda20_50.json.

2. Modify loop.py to Pass the Iteration Number

Now in loop.py, we modify it to pass the iteration number to grploop1.py and run it multiple times:

import subprocess

# Loop for 20 iterations
for i in range(1, 21):  # Loops 1 to 20
    # Call grploop1.py with the current iteration number
    subprocess.call(['python', 'grploop1.py', str(i)])

Key Changes:

  • subprocess.call(['python', 'grploop1.py', str(i)]) passes the current iteration number i to grploop1.py.

Expected Output:

For ids = ('5cda', '6cda'), running loop.py will produce 40 JSON files:

  1. Files for 5cda:

    • 5cda1_1.json, 5cda1_2.json, ..., 5cda1_50.json for iteration 1.
    • 5cda2_1.json, 5cda2_2.json, ..., 5cda2_50.json for iteration 2.
    • ...
    • 5cda20_1.json, 5cda20_2.json, ..., 5cda20_50.json for iteration 20.

  2. Files for 6cda:

    • 6cda1_1.json, 6cda1_2.json, ..., 6cda1_50.json for iteration 1.
    • 6cda2_1.json, 6cda2_2.json, ..., 6cda2_50.json for iteration 2.
    • ...
    • 6cda20_1.json, 6cda20_2.json, ..., 6cda20_50.json for iteration 20.

Explanation:

  • grploop1.py now accepts an iteration argument and appends it to the file name to create distinct files for each loop iteration.
  • The loop.py script runs the grploop1.py script 20 times, passing the current iteration number so that the filenames are generated accordingly.

Conclusion:

This approach will generate 40 unique files (20 for each user_id), following the pattern you've requested (5cda1.json, 5cda2.json, ..., 6cda1.json, ..., 6cda20.json). Each file will be saved based on the loop number and the user ID, and you can ensure that each execution's output is correctly saved and identified.

#py