Bix CSV extract filter special characters out
We have a BIX csv extract process running today. It works well but we have some fields that are in the BLOB column that are free form. Agents will paste text in from other tools that include special characters such as carriage return/linefeed. This breaks the import of the CSV file into downstream systems. Are there built in BIX filters that can be applied? We're looking at shell scripts/awk/python today as well.
Maantic Technologies Pvt Ltd
US
Hi @DPCLARK12
Hope you are doing well.
BIX does not have built-in filters specifically designed to handle special characters in BLOB columns. However, there are some general filters that may be useful:
Data Type Filter: You can filter the data by data type, e.g., `data_type = "BLOB"`. Contains Filter: You can check if the BLOB field contains a specific substring, e.g., `contains("carriage return")`. However, this may not be effective if the special characters are not encoded as literal characters.
Can you please let me know what you meant by "We're looking at shell scripts/awk/python today as well."
Hope it helps.
Thanks
Megha