Process Consumed Messages from the Queue

This is the default handler for processing messages consumed from the aggregration message broker. At the conceptual level, all network interface capture tools forward JSON dictionaries to this class.

class network_pipeline.record_packets_to_csv.RecordPacketsToCSV[source]
build_all_keys_dict()[source]
build_flat_msg(id=None, msg=None)[source]
Parameters:
  • id – unique id for this message
  • msg – message dictionary to flatten
convert_to_df()[source]
create_json_archive()[source]
flatten_all()[source]
handle_msg(body, org_message)[source]
Parameters:
  • body – dictionary contents from the message body
  • org_message – message object can ack, requeue or reject
process_arp_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – arp frame for packet
process_dns_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – dns frame for packet
process_ether_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – ether frame for packet
process_icmp_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – icmp frame for packet
process_ip_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – ip frame for packet
process_ipvsix_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – ipv6 frame for packet
process_pad_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – pad frame for packet
process_raw_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – raw frame for packet
process_tcp_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – tcp frame for packet
process_udp_frame(id=None, msg=None)[source]

Convert a complex nested json dictionary to a flattened dictionary and capture all unique keys for table construction

Parameters:
  • id – key for this msg
  • msg – udp frame for packet
publish_predictions_to_core()[source]
save_data()[source]
save_df_as_csv()[source]
write_to_file(data_dict, output_file_path)[source]
Parameters:
  • data_dict –
  • output_file_path –