mycelium.Inchworm()

Here is the class graph organized into a markdown table:

Method	Description
url_to_html(recursion=true/false)	Converts "url" to raw HTML, images, and meta
html_to_json(html)	(Optional) Converts HTML to JSON
render_webpage(json)	Converts JSON to HTML

mycelium.Data.graph()

This is our Python Guide for ML development. See Table 1. Below

graph.html_to_json(html: String)

This method is used to convert html format into json format. To use this method we are calling "http://199.88.191.194:5008" server. It will convert the html format into json format.

Parameter

The html passed in the function is a string representing html format from which you want to retrieve the json content.

Example:

To use please refer below example

    
    

  
from mycelium import data
html = '<h1>Hello World</h1>'
graph.html_to_json(html)

graph.json_to_html()

This method is used to convert json format into html format. To use this method we are calling "http://199.88.191.194:5008" server. It will convert the json format into html format.

Parameter

    
    

  
json: String

The json passed in the function is a string representing json format from which you want to retrieve the html content.

Example:

To use please refer below example

    
    

  
from mycelium import data
json = {
  'age':22
  }
graph.json_to_html(json)

graph.get_url_to_html()

This method is used to get html of given url. To use this method we are calling "http://199.88.191.194:5008" server. It will convert the given url's html into our html format.

Parameter

    
    

  
URL: String

The URL passed in the function is a string representing a web URL from which you want to retrieve the HTML content.

Example:

To use please refer below example

    
    

  
from mycelium import data
URL= 'https://dev-cloud.virtuousai.com/meta.json'
graph.get_url_to_html(URL)

graph.get_url_to_json()

This method is used to get json of given url. To use this method we are calling "http://199.88.191.194:5008" server. It will convert the given url's json into our json format.

Parameter

    
    

  
URL: String

The URL passed in the function is a string representing a web URL from which you want to retrieve the JSON content.

Example:

To use please refer below example:

    
    

  
from mycelium import data
URL = 'https://www.google.com'
graph.get_url_to_json(URL)

graph.store_url_to_html()

This method is used to store html of given url. It will store html return by url into input_dir. To use this method we are calling "http://199.88.191.194:5008" server.

Parameter

    
    

  
URL: String

The URL passed in the function is a string representing a web URL from which you want to retrieve the HTML content

    
    

  
output_dir: String

The output_dir is a string representing the directory where you want to store the HTML file obtained from the URL.

Example:

To use please refer below example:

    
    

  
from mycelium import data
url = 'https://dev-cloud.virtuousai.com/meta.json'
output_dir =html/vai.html
graph.store_url_to_html(url, output_dir)

graph.store_url_files_to_html()

This method is used to storing html of stored urls json file on given path. It will store html return by url given in input_dir and store in output_dir. To use this method we are calling "http://199.88.191.194:5007" server.

Parameter

    
    

  
input_dir: String

The input_dir is a string representing the directory where the stored URLs JSON file is located.

    
    

  
output_dir: String

The output_dir is a string representing the directory where you want to store the HTML files obtained from the URLs specified in the JSON file.

    
    

  
chunk_size:Number

In order to prevent overloading the main PC, we are breaking the file into small chunks and implementing a 15-second delay between downloads. This approach ensures proper results and determines the size of each individual chunk.

Example:

To use please refer below example:

    
    

  
from mycelium import data
input_dir = "DATA/COMMON-CRAWL/c4repset/url_list/c4-train.ranking_v1_1GB.urllist.json"
output_dir = "DATA/COMMON-CRAWL/c4repset/output_new"
chunk_size = "1000"
generate = "HTML"
graph.store_url_files_to_html(input_dir, output_dir, chunk_size, generate)

graph.convert_text_to_json()

This method is used to convert text file into json file. To use this method we are calling "http://199.88.191.194:5007" server. It will convert text file into json file.

Parameter

    
    

  
input: String

The input is a string representing the directory where the stored URLs JSON file is located.

    
    

  
output: String

The output is a string representing the JSON file name you want to store.

Example:

To use please refer below example:

    
    

  
from mycelium import data
input = "./input.txt"
output = "./input.json"
graph.convert_text_to_json(input,output)