Skip to content

Python Guide

This is our Python Guide for ML development. See Table 1. Below


Table 1. vai_utils.Data.graph()

Here are the methods and their descriptions organized into markdown tables:

1. mycelium.data

Method Description
Dataset.crop_images() This method is used to crop the images.
Dataset.collage_images() This method is used to collage the images.
Method Description
graph.html_to_json() This method is used to convert html format into json format.
graph.json_to_html() This method is used to convert json format into html format.
graph.get_url_to_html() This method is used to get html of given url.
graph.get_url_to_json() This method is used to get json of given url.
graph.store_url_to_html() This method is used to store html of given url.
graph.store_url_files_to_html() This method is used to storing html of stored urls json file on given path.
graph.convert_text_to_json() This method is used to convert text file into json file.


Table 1. vai_utils.Data.image()

Method Description
images.crop_images() This method is used to crop the images.
images.collage_images() This method is use to the college images.

vai_utils.data()

Data.html_to_json()

This method is used to convert html format into json format. To use this method we are calling "http://199.88.191.194:5008" server. It will convert the html format into json format.

Parameter


    
    

  
html: String

The html passed in the function is a string representing html format from which you want to retrieve the json content.

Example:

To use please refer below example


    
    

  
from vai_utils import data
html = '<h1>Hello World</h1>'
Data.html_to_json(html)



Data.json_to_html()

This method is used to convert json format into html format. To use this method we are calling "http://199.88.191.194:5008" server. It will convert the json format into html format.

Parameter


    
    

  
json: String

The json passed in the function is a string representing json format from which you want to retrieve the html content.

Example:

To use please refer below example


    
    

  
from vai_utils import data
json = {
  'age':22
  }
Data.json_to_html(json)



Data.get_url_to_html()

This method is used to get html of given url. To use this method we are calling "http://199.88.191.194:5008" server. It will convert the given url's html into our html format.

Parameter


    
    

  
URL: String

The URL passed in the function is a string representing a web URL from which you want to retrieve the HTML content.

Example:

To use please refer below example


    
    

  
from vai_utils import data
URL= 'https://dev-cloud.virtuousai.com/meta.json'
Data.get_url_to_html(URL)



Data.get_url_to_json()

This method is used to get json of given url. To use this method we are calling "http://199.88.191.194:5008" server. It will convert the given url's json into our json format.

Parameter


    
    

  
URL: String

The URL passed in the function is a string representing a web URL from which you want to retrieve the JSON content.

Example:

To use please refer below example:


    
    

  
from vai_utils import data
URL = 'https://www.google.com'
Data.get_url_to_json(URL)



Data.crop_images()

This method is used to crop the images. It will crop images, given that path of images, and height of width of that images. To use this method we are calling "http://199.88.191.194:5008" server.

Parameter


    
    

  
path: String

path: String
path: String

The path is a string representing the path of the image file that you want to crop


    
    

  
height: Integer

Height is an integer specifying the desired height of the cropped image


    
    

  
width: Integer

Width is an integer specifying the desired width of the cropped image

Example:

To use please refer below example:


    
    

  
from vai_utils import data
path = '.image/dog.png'
height = 300
width = 500
Data.crop_images(path,height, width)



Data.collage_images()

This method is use to the college images. It will college images, given that path of images, and height of width of that images. To use this method we are calling "http://199.88.191.194:5007" server.

Parameter


    
    

  
path: String

The path is a string representing the path of the images that you want to create a collage


    
    

  
height: Integer

Height is an integer specifying the desired height of each image in the collage


    
    

  
width: Integer

Width is an integer specifying the desired width of each image in the collage

Example:

To use please refer below example:


    
    

  
from vai_utils import data
path = '.image/cat.png'
height = 300
width = 500
Data.collage_images(path,height, width)



Data.store_url_to_html()

This method is used to store html of given url. It will store html return by url into input_dir. To use this method we are calling "http://199.88.191.194:5008" server.

Parameter


    
    

  
URL: String

The URL passed in the function is a string representing a web URL from which you want to retrieve the HTML content


    
    

  
output_dir: String

The output_dir is a string representing the directory where you want to store the HTML file obtained from the URL.

Example:

To use please refer below example:


    
    

  
from vai_utils import data
url = 'https://dev-cloud.virtuousai.com/meta.json'
output_dir =html/vai.html
Data.store_url_to_html(url, output_dir)



Data.store_url_files_to_html()

This method is used to storing html of stored urls json file on given path. It will store html return by url given in input_dir and store in output_dir. To use this method we are calling "http://199.88.191.194:5007" server.

Parameter


    
    

  
input_dir: String

The input_dir is a string representing the directory where the stored URLs JSON file is located.


    
    

  
output_dir: String

The output_dir is a string representing the directory where you want to store the HTML files obtained from the URLs specified in the JSON file.


    
    

  
chunk_size:Number

In order to prevent overloading the main PC, we are breaking the file into small chunks and implementing a 15-second delay between downloads. This approach ensures proper results and determines the size of each individual chunk.

Example:

To use please refer below example:


    
    

  
from vai_utils import data
input_dir = "DATA/COMMON-CRAWL/c4repset/url_list/c4-train.ranking_v1_1GB.urllist.json"
output_dir = "DATA/COMMON-CRAWL/c4repset/output_new"
chunk_size = "1000"
generate = "HTML"
Data.store_url_files_to_html(input_dir, output_dir, chunk_size, generate)

Data.convert_text_to_json()

This method is used to convert text file into json file. To use this method we are calling "http://199.88.191.194:5007" server. It will convert text file into json file.

Parameter


    
    

  
input: String

The input is a string representing the directory where the stored URLs JSON file is located.


    
    

  
output: String

The output is a string representing the JSON file name you want to store.

Example:

To use please refer below example:


    
    

  
from vai_utils import data
input = "./input.txt"
output = "./input.json"
Data.convert_text_to_json(input,output)