Yugiquery Module

This module contains the functions used to produce the YugiQuery reports.

class yugiquery.yugiquery.CG

Bases: Enum

Enum representing the card game formats.

CG

Both TCG and OCG.

Type:

str

ALL

Alias for CG, representing all card games.

Type:

CG

BOTH

Alias for CG, representing both card games.

Type:

CG

TCG

The ‘trading card game’ type.

Type:

str

OCG

The ‘official card game’ type.

Type:

str

yugiquery.yugiquery.benchmark(timestamp: Arrow, report: str | None = None) → None

Records the execution time of a report and saves the data to a JSON file.

Parameters:

  • timestamp (arrow.Arrow) – The timestamp when the report execution began.

  • report (str | None, optional) – The name of the report being benchmarked. If None, tries obtaining report name from JPY_SESSION_NAME environment variable.

Returns:

None

yugiquery.yugiquery.card_properties = {'bandai': ['card_type', 'level', 'atk', 'def', 'number', 'monster_type', 'rule', 'sets', 'rarity', 'ability', 'date'], 'counter': ['password', 'card_type', 'effect_type', 'archseries', 'alternate_artwork', 'edited_artwork', 'tcg', 'ocg', 'date'], 'monster': ['password', 'card_type', 'primary', 'secondary', 'attribute', 'monster_type', 'stars', 'atk', 'def', 'scale', 'link', 'arrows', 'effect_type', 'archseries', 'alternate_artwork', 'edited_artwork', 'tcg', 'ocg', 'date'], 'rush': ['card_type', 'property', 'primary', 'attribute', 'monster_type', 'stars', 'atk', 'def', 'effect_type', 'archseries', 'date', 'rush_alt_artwork', 'rush_edited_artwork', 'maximum_atk', 'misc'], 'skill': ['card_type', 'property', 'archseries', 'tcg', 'date', 'speed', 'character'], 'speed': ['password', 'card_type', 'property', 'primary', 'secondary', 'attribute', 'monster_type', 'stars', 'atk', 'def', 'effect_type', 'archseries', 'alternate_artwork', 'edited_artwork', 'tcg', 'ocg', 'date', 'speed'], 'st': ['password', 'card_type', 'property', 'effect_type', 'archseries', 'alternate_artwork', 'edited_artwork', 'tcg', 'ocg', 'date']}

A dictionary mapping card types to their corresponding properties to query.

yugiquery.yugiquery.card_query(*args, **kwargs) → str

Builds a query string to be passed to the yugipedia Wiki API for a card search query.

Parameters:

  • default (bool, optional) – The default card query string, containing all properties for Monster, Spell and Trap cards. Defaults to False.

  • *args – Properties to get from the prop_dict or to use directly as query string if not in prop_dict.

  • **kwargs – Properties to include, if True, or remove, if False, from the query string.

Raises:

ValueError – If default is not a valid card type.

Returns:

A string containing the arguments to be passed to the API for the card search query.

Return type:

str

yugiquery.yugiquery.cleanup_data(dry_run=False) → None

Cleans up data files, keeping only the most recent file from each month and week.

Parameters:

dry_run (bool) – If True, the function will only print the files that would be deleted without actually deleting them. Defaults to False.

Returns:

None

yugiquery.yugiquery.condense_benchmark(benchmark: Dict[str, List[Dict[str, str | float]]]) → Dict[str, List[Dict[str, str | float]]]

Condenses a benchmark dictionary by calculating the weighted average and total weight for each key.

Parameters:

benchmark (Dict[str, List[Dict[str, str | float]]]) – A dictionary containing benchmark data.

Returns:

The condensed benchmark dictionary with updated entries.

Return type:

Dict[str, List[Dict[str, str | float]]]

yugiquery.yugiquery.condense_changelogs(files: DataFrame) → Tuple[DataFrame, str]

Condenses multiple changelog files into a consolidated dataframe and generates a new filename.

Parameters:

files (pd.DataFrame) – A dataframe containing the changelog files.

Returns:

A tuple containing the consolidated changelog dataframe and the new filename.

Return type:

Tuple[pd.DataFrame, str]

yugiquery.yugiquery.export_notebook(input_path: str | None = None, output_path: str | None = None, template: str = 'auto', no_input: bool = True) → None

Convert a Jupyter notebook to HTML using nbconvert and save the output to disk.

Parameters:

  • input_path (str | None, optional) – The path to the Jupyter notebook file to convert. If None, gets the notebook path with get_notebook_path. Defaults to None.

  • output_path (str | None, optional) – The path to save the converted HTML file. If None, saves the file to the REPORTS directory. Defaults to None.

  • template (str, optional) – The name of the nbconvert template to use. If “auto”, uses “labdynamic” if available, otherwise uses “lab”. Defaults to “auto”.

  • no_input (bool, optional) – If True, excludes input cells from the output. Defaults to True.

Raises:

ValueError – If no notebook path is provided and cannot be found with get_notebook_path.

Returns:

None

yugiquery.yugiquery.fetch_all_set_lists(cg: CG = CG.CG, step: int = 40, **kwargs) → DataFrame

Fetches all set lists for a given card game.

Parameters:

  • cg (CG, optional) – The card game to fetch set lists for. Defaults to CG.ALL.

  • step (int, optional) – The number of sets to fetch at once. Defaults to 50.

  • **kwargs – Additional keyword arguments to pass to fetch_set_list_pages and fetch_set_lists.

Returns:

A DataFrame containing all set lists for the specified card game.

Return type:

pd.DataFrame

Raises:

Any exceptions raised by fetch_set_list_pages() or fetch_set_lists(). –

yugiquery.yugiquery.fetch_bandai(bandai_query: str | None = None, limit: int = 200, **kwargs) → DataFrame

Fetch Bandai cards.

Parameters:

  • bandai_query (str | None, optional) – A string representing a SMW query to search for. Defaults to None.

  • limit (int, optional) – An integer that represents the maximum number of results to fetch. Defaults to 200.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A pandas DataFrame object containing the properties of the fetched Bandai cards.

Return type:

pandas.DataFrame

yugiquery.yugiquery.fetch_counter(counter_query: str | None = None, cg=CG.CG, step: int = 500, limit: int = 5000, **kwargs) → DataFrame

Fetch counter cards based on query and properties of the cards.

Parameters:

  • counter_query (str | None, optional) – A string representing a SMW query to search for. Defaults to None.

  • step (int, optional) – An integer that represents the number of results to fetch at a time. Defaults to 500.

  • limit (int, optional) – An integer that represents the maximum number of results to fetch. Defaults to 5000.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A pandas DataFrame object containing the properties of the fetched counter cards.

Return type:

pandas.DataFrame

Raises:

ValueError – Raised if the “cg” argument is not a valid CG.

yugiquery.yugiquery.fetch_errata(errata: str = 'all', step: int = 500, **kwargs) → DataFrame

Fetches errata information from the yuipedia Wiki API.

Parameters:

  • errata (str) – The type of errata information to fetch. Valid values are ‘name’, ‘type’, and ‘all’. Defaults to ‘all’.

  • step (int) – The number of results to fetch in each API call. Defaults to 500.

  • **kwargs – Additional keyword arguments to pass to fetch_categorymembers.

Returns:

A pandas DataFrame containing a boolean table indicating whether each card has errata information for the specified type.

Return type:

pandas.DataFrame

yugiquery.yugiquery.fetch_monster(monster_query: str | None = None, cg: CG = CG.CG, step: int = 500, limit: int = 5000, exclude_token=True, **kwargs) → DataFrame

Fetch monster cards based on query and properties of the cards.

Parameters:

  • monster_query (str | None, optional) – A string representing a SMW query to search for. Defaults to None.

  • cg (CG, optional) – An Enum that represents the card game to fetch cards from. Defaults to CG.ALL.

  • step (int, optional) – An integer that represents the number of results to fetch at a time. Defaults to 500.

  • limit (int, optional) – An integer that represents the maximum number of results to fetch. Defaults to 5000.

  • exclude_token (bool, optional) – A boolean that determines whether to exclude Monster Tokens or not. Defaults to True.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A pandas DataFrame object containing the properties of the fetched monster cards.

Return type:

pandas.DataFrame

Raises:

ValueError – Raised if the “cg” argument is not a valid CG.

yugiquery.yugiquery.fetch_rush(rush_query: str | None = None, step: int = 500, limit: int = 5000, **kwargs) → DataFrame

Fetches Rush Duel cards from the Yu-Gi-Oh! Wikia API.

Parameters:

  • rush_query (str | None, optional) – A search query to filter the results. If not provided, it defaults to “rush”.

  • step (int, optional) – The number of results to fetch in each API call. Defaults to 500.

  • limit (int, optional) – The maximum number of results to fetch. Defaults to 5000.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A pandas DataFrame containing the fetched Rush Duel cards.

yugiquery.yugiquery.fetch_set_list_pages(cg: CG = CG.CG, step: int = 500, limit=5000, **kwargs) → DataFrame

Fetches a list of ‘Set Card Lists’ pages from the yugipedia Wiki API.

Parameters:

  • cg (CG) – A member of the CG enum representing the card game for which set lists are being fetched.

  • step (int) – The number of pages to fetch in each API request.

  • limit (int) – The maximum number of pages to fetch.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A DataFrame containing the titles of the set list pages.

Return type:

pd.DataFrame

Raises:

ValueError – Raised if the “cg” argument is not a valid CG.

yugiquery.yugiquery.fetch_skill(skill_query: str | None = None, step: int = 500, limit: int = 5000, **kwargs) → DataFrame

Fetches skill cards from the yugipedia Wiki API.

Parameters:

  • skill_query (str | None, optional) – A string representing a SMW query to search for. Defaults to None.

  • step (int, optional) – The number of results to fetch in each API call. Defaults to 500.

  • limit (int, optional) – The maximum number of results to fetch. Defaults to 5000.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A pandas DataFrame containing the fetched skill cards.

yugiquery.yugiquery.fetch_speed(speed_query: str | None = None, step: int = 500, limit: int = 5000, **kwargs) → DataFrame

Fetches TCG Speed Duel cards from the yugipedia Wiki API.

Parameters:

  • speed_query (str | None, optional) – A string representing a SMW query to search for. Defaults to None.

  • step (int, optional) – The number of results to fetch in each API call. Defaults to 500.

  • limit (int, optional) – The maximum number of results to fetch. Defaults to 5000.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A pandas DataFrame containing the fetched TCG Speed Duel cards.

yugiquery.yugiquery.fetch_st(st_query: str | None = None, st: str = 'both', cg: CG = CG.CG, step: int = 500, limit: int = 5000, **kwargs) → DataFrame

Fetch spell or trap cards based on query and properties of the cards.

Parameters:

  • st_query (str | None, optional) – A string representing a SMW query to search for. Defaults to None.

  • st (str, optional) – A string representing the type of cards to fetch, either “spell”, “trap”, “both”, or “all”. Defaults to “both”.

  • cg (CG, optional) – An Enum that represents the card game to fetch cards from. Defaults to CG.ALL.

  • step (int, optional) – An integer that represents the number of results to fetch at a time. Defaults to 500.

  • limit (int, optional) – An integer that represents the maximum number of results to fetch. Defaults to 5000.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A pandas DataFrame object containing the properties of the fetched spell/trap cards.

Return type:

pandas.DataFrame

Raises:

  • ValueError – Raised if the “st” argument is not one of “spell”, “trap”, “both”, or “all”.

  • ValueError – Raised if the “cg” argument is not a valid CG.

yugiquery.yugiquery.fetch_token(token_query: str | None = None, cg=CG.CG, step: int = 500, limit: int = 5000, **kwargs) → DataFrame

Fetch token cards based on query and properties of the cards.

Parameters:

  • token_query (str | None, optional) – A string representing a SWM query to search for. Defaults to None.

  • step (int, optional) – An integer that represents the number of results to fetch at a time. Defaults to 500.

  • limit (int, optional) – An integer that represents the maximum number of results to fetch. Defaults to 5000.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A pandas DataFrame object containing the properties of the fetched token cards.

Return type:

pandas.DataFrame

Raises:

ValueError – Raised if the “cg” argument is not a valid CG.

yugiquery.yugiquery.fetch_unusable(query: str | None = None, cg: CG = CG.CG, filter=True, step: int = 500, limit: int = 5000, **kwargs) → DataFrame

Fetch unusable cards based on query and properties of the cards. Unusable cards include “Strategy cards”, “Tip cards”, “Card Checklists”, etc, which are not actual cards. The filter option enables filtering those out and keeping only cards such as Duelist Kingdom “Ticket cards”, old video-game promo “Character cards” and “Non-game cards” which have the layout of a real card, such as “Everyone’s King”. This criteria is not free of ambiguity.

Parameters:

  • query (str | None, optional) – A string representing a SMW query to search for. Defaults to None.

  • cg (CG, optional) – An Enum that represents the card game to fetch cards from. Defaults to CG.ALL.

  • filter (bool, optional) – Keep only “Character Cards”, “Non-game cards” and “Ticket Cards”.

  • step (int, optional) – An integer that represents the number of results to fetch at a time. Defaults to 500.

  • limit (int, optional) – An integer that represents the maximum number of results to fetch. Defaults to 5000.

  • **kwargs – Additional keyword arguments to pass to fetch_properties.

Returns:

A pandas DataFrame object containing the properties of the fetched spell/trap cards.

Return type:

pandas.DataFrame

Raises:

ValueError – Raised if the “cg” argument is not a valid CG.

yugiquery.yugiquery.footer(timestamp: Arrow | None = None) → Markdown

Generates a Markdown footer with a timestamp. If there is no footer.md file in the ASSETS directory, prints error message and an returns None.

Parameters:

timestamp (arrow.Arrow | None, optional) – The timestamp to use. If None, uses the current time. Defaults to None.

Returns:

The generated Markdown footer.

Return type:

Markdown

yugiquery.yugiquery.format_artwork(row: Series) → Tuple[str]

Formats a row of a dataframe that contains “alternate artworks” and “edited artworks” columns. If the “alternate artworks” column(s) in the row contain at least one “True” value, adds “Alternate” to the result tuple. If the “edited artworks” column(s) in the row contain at least one “True” value, adds “Edited” to the result tuple. Returns the result tuple. :param row: A row of a dataframe that contains “alternate artworks” and “edited artworks” columns. :type row: pd.Series

Returns:

The formatted row as a tuple.

Return type:

Tuple[str]

yugiquery.yugiquery.format_errata(row: Series) → Tuple[str]

Formats errata information from a pandas Series and returns a tuple of errata types.

Parameters:

row (pd.Series) – A pandas Series containing errata information for a single card.

Returns:

Tuple of errata types if any errata information is present in the input Series, otherwise np.nan.

Return type:

Tuple[str]

yugiquery.yugiquery.generate_changelog(previous_df: DataFrame, current_df: DataFrame, col: str | List[str]) → DataFrame

Generates a changelog DataFrame by comparing two DataFrames based on a specified column.

Parameters:

  • previous_df (pd.DataFrame) – A DataFrame containing the previous version of the data.

  • current_df (pd.DataFrame) – A DataFrame containing the current version of the data.

  • col (str | List[str]) – The name of the column to compare the DataFrames on.

Returns:

A DataFrame containing the changes made between the previous and current versions of the data. The DataFrame will have the following columns: the specified column name, the modified data, and the indicator for whether the data is new or modified renamed as version (either “Old” or “New”). If there are no changes, the function will return a DataFrame with no rows.

Return type:

pd.DataFrame

yugiquery.yugiquery.get_notebook_path() → Path

Gets the path of the current notebook opened in JupyterLab. If the path cannot be obtained, returns None.

Parameters:

None

Returns:

The path of the current notebook.

Return type:

Path

yugiquery.yugiquery.header(name: str | None = None) → Markdown

Generates a Markdown header with a timestamp and the name of the notebook (if provided). If there is no header.md file in the ASSETS directory, prints an error message and returns None.

Parameters:

name (str | None, optional) – The name of the notebook. If None, attempts to extract the name from the environment variable JPY_SESSION_NAME. Defaults to None.

Returns:

The generated Markdown header.

Return type:

Markdown

yugiquery.yugiquery.load_corrected_latest(name_pattern: str, tuple_cols: List[str] = ['Secondary type', 'Effect type', 'Link Arrows', 'Archseries', 'Artwork', 'Errata', 'Rarity', 'Cover card']) → Tuple[DataFrame, Arrow] | Tuple[None, None]

Loads the most recent data file matching the specified name pattern and applies corrections.

Parameters:

  • name_pattern (str) – Data file name pattern to load.

  • tuple_cols (List[str]) – List of columns containing tuple values to apply literal_eval. Defaults to [“Secondary type”, “Effect type”, “Link Arrows”, “Archseries”, “Artwork”, “Errata”, “Rarity”, “Cover card”].

Returns:

A tuple containing the loaded dataframe and the timestamp of the file.

Return type:

Tuple[pd.DataFrame, arrow.Arrow]

yugiquery.yugiquery.merge_errata(input_df: DataFrame, input_errata_df: DataFrame) → DataFrame

Merges errata information from an input errata DataFrame into an input DataFrame based on card names.

Parameters:

  • input_df (pd.DataFrame) – A pandas DataFrame containing card information.

  • input_errata_df (pd.DataFrame) – A pandas DataFrame containing errata information.

Returns:

A pandas DataFrame with errata information merged into it.

Return type:

pd.DataFrame

yugiquery.yugiquery.merge_set_info(input_df: DataFrame, input_info_df: DataFrame) → DataFrame

Merges set information from an input set info DataFrame into an input set list DataFrame based on set and region.

Parameters:

  • input_df (pd.DataFrame) – A pandas DataFrame containing set lists.

  • input_info_df (pd.DataFrame) – A pandas DataFrame containing set information.

Returns:

A pandas DataFrame with set information merged into it.

Return type:

pd.DataFrame

Raises:

ValueError – If the input DataFrame does not contain the required “Set” and “Region” columns.

yugiquery.yugiquery.run(reports: str | List[str] = 'all', progress_handler: ProgressHandler | None = None, cleanup: bool | Literal['auto'] = 'auto', dry_run: bool = False, squash: bool = True, discord: bool | Namespace = False, telegram: bool | Namespace = False, debug: bool = False) → None

Executes all notebooks in the user and package NOTEBOOKS directories that match the specified report, updates the page index to reflect the last execution timestamp, and clean up redundant data files.

Parameters:

  • reports (str | List[str], optional) – The report to generate. Defaults to ‘all’.

  • progress_handler (ProgressHandler | None, optional) – An optional ProgressHandler instance to report execution progress. Defaults to None.

  • cleanup (bool | Literal["auto"], optional) – whether to cleanup data files after execution. If True, perform cleanup, if False, doesn’t perform cleanup. If ‘auto’, performs cleanup if there are more than 4 data files for each report (assuming one per week). Defaults to ‘auto’.

  • dry_run (bool, optional) – dry_run flag to pass to cleanup_data method call. Defaults to False.

  • squash (bool, optional) – squash commits after execution. Defaults to True.

  • discord (bool | argparse.Namespace, optional) – Discord configuration, either as a boolean or argparse.Namespace. Default is False.

  • telegram (bool | argparse.Namespace, optional) – Telegram configuration, either as a boolean or argparse.Namespace. Default is False.

  • debug (bool, optional) – Whether to enable debug mode. Default is False.

Raises:

Exception – Raised if any exceptions occur during notebook execution.

Returns:

This function does not return a value.

Return type:

None

yugiquery.yugiquery.run_notebooks(reports: str | list[str], external_pbar: tqdm_asyncio | None = None, discord: bool | Namespace = False, telegram: bool | Namespace = False, dry_run: bool = False, debug: bool = False) → None

Execute specified Jupyter notebooks using Papermill.

Parameters:

  • reports (str | List[str]) – List of notebooks to execute.

  • external_pbar (tqdm | None, optional) – An external tqdm progress bar to update. Defaults to None.

  • discord (bool | argparse.Namespace, optional) – Discord configuration, either as a boolean or argparse.Namespace. Default is False.

  • telegram (bool | argparse.Namespace, optional) – Telegram configuration, either as a boolean or argparse.Namespace. Default is False.

  • dry_run (bool, optional) – Whether to run in dry run mode. Default is False.

  • debug (bool, optional) – Whether to enable debug mode. Default is False.

Returns:

None

Raises:

Exception – Raised if any exceptions occur during notebook execution.

yugiquery.yugiquery.save_notebook() → None

Save the current notebook opened in JupyterLab to disk.

Parameters:

None

Returns:

None

yugiquery.yugiquery.update_index(dry_run: bool = False) → str

Update the index.md and README.md files with a table of links to all HTML reports in the REPORTS directory. Also update the @REPORT_|_TIMESTAMP@ and @TIMESTAMP@ placeholders in the index.md file with the latest timestamp. If the update is successful, commit the changes to Git with a commit message that includes the timestamp. If there is no index.md or README.md files in the ASSETS directory, print an error message and abort.

Parameters:

dry_run (bool, optional) – If True, the function will not commit the changes to Git. Defaults to False.

Returns:

The result of the Git commit if not dry_run, otherwise advisory message.

Return type:

str

yugiquery.yugiquery.update_rarities(save: bool = True) → Dict[str, str]

Updates the rarities.json file by fetching redirects and backlinks for the rarity codes and names.

Parameters:

save (bool, optional) – Whether to save the new dictionary. Defaults to True.

Returns:

The updated rarities dictionary.

Return type:

Dict[str, str]

yugiquery.yugiquery.update_regions(save: bool = True) → Dict[str, str]

Updates the regions.json file by fetching backlinks and redirects for the region names and codes. Note: The category “Termonology” holds the regions in the wiki.

Parameters:

save (bool, optional) – Whether to save the new dictionary. Defaults to True.

Returns:

The updated regions dictionary.

Return type:

Dict[str, str]