“EBCDIC” is an abbreviation of “Extended Binary Coded Decimal Interchange Code”. It's a binary encoding for displayable characters on IBM and other mainframe and mid-size computers used by governments and other large institutions, such as mortgage companies and banks.
Today’s personal computers use a different binary encoding for displayable characters called “ASCII” (American Standard Code for Information Interchange).
The two encodings are very different. EBCDIC dates back to the use of computer punch cards, which had a different set of displayable characters than today’s PCs. Also, when EBCDIC was defined, computer memory and storage were very limited and very expensive. Programmers figured out ingenious ways to conserve space, such as “packing” information into fewer bytes where possible.
Because of their differences, converting a file between EBCDIC and ASCII or other formats usually requires specialized software or a file editor like vEdit that understands how to do the conversion.
While most modern computers use the ASCII character set, early IBM (and other) mainframes used a different set, called EBCDIC, used to program computers in the mid-late 20th century.
If the EBCDIC file contains only text, it can easily be converted byte-for-byte to ASCII; some ftp programs will even do this.
However, much more complex EBCDIC data files are actually databases, often created by COBOL programs. They could also be created in IBM 360 assembly language and other old languages. These data files can include compressed binary fields to represent numbers. For example, a signed 7-digit number is compressed into just 4 bytes. Therefore, a byte-for-byte conversion is not possible. The conversion process needs to know the position, size and type of compressed numbers.
EBCDIC data files are typically supplied with documentation that describes the size and type of each field. Some, like data dictionaries, may include positional information. The most common and preferred layout specification is called a "COBOL Copybook". The supplier of the data file generally has the copybook and can provide it with the data.
EBCDIC data files can have a very complex structure, much more complex than any modern database table. Without the layout information, the EBCDIC data file can be very difficult to understand.
For example, EBCDIC data can include multiple record types of variable length. Imagine taking a dozen different SQL tables, merging them and shuffling the records; that is the structure of an EBCDIC file with multiple record types. A single variable length EBCDIC record could consist of a bank customer's basic information plus all the deposits and withdrawals they made in one month. A single EBCDIC record can consist of a "master" record and a variable number of "detail" records. Therefore, a single EBCDIC data file often needs to be converted into multiple ASCII files, perhaps with the addition of relational keys.
If you need data from a company or government agency that uses a mainframe or midrange computer, they might send you files in EBCDIC format.
Before you can load the data into a spreadsheet like Excel or a relational database like SQL Server on your PC, you must convert the data to ASCII.
Some providers will convert their data for you, but that can be very expensive.
EBCDIC and ASCII are very different formats. Converting between them is usually not straightforward. There are several reasons for this. For example:
- Different Character Mapping
- The same displayable characters are mapped to different binary (machine) values. For example, the letter “A” in EBCDIC is decimal 193 (hex xC1). In ASCII, “A” is decimal 65 (hex x41).
- Delimiters
- ASCII uses special characters ("delimiters") to mark the end of each line in the file and to separate the words and numbers in each line. EBCDIC does not use delimiters. Instead, you need a separate "layout file" to figure out where each field and each line ends.
- Packed Fields
- Many EBCDIC files use a scheme called "packing" to reduce the amount of space it takes to represent numbers. Each digit (0-9) of a number is represented by 4-bits (a half-byte, or "nibble"), so each byte can hold 2 digits. The layout file tells you which fields are packed and how. So it takes special software like the vEdit EBCDIC Converter Packages to translate an EBCDIC file with packed fields to ASCII.
Many EBCDIC data files contain other types of special fields that required special software to convert them.
This is just a summary of the differences. For more details, see our EBCDIC vs. ASCII discussion.
That depends on how complicated your EBCDIC file is. Some free conversion tools exist on the internet, but they can only handle the very simplest layout and don't work for most real-world data. You'll probably need to buy a file editor designed for the task to convert it yourself or hire a professional to do it for you.
The simplest EBCDIC file has fixed-length records and does not use packed fields. As long as you know the layout, a file editor like vEdit, with the ability to convert between formats (e.g., EBCDIC, ASCII, HEX) can do the conversion without much effort.
Converting an EBCDIC file with packed BCD, zoned, signed, binary and other special fields to ASCII requires specialized software that has information about the various fields. Otherwise, any character-by-character EBCDIC to ASCII translation software or hardware will corrupt these fields and the original data will no longer be recoverable.
You may find that it's not worth your time or investment to do the conversion yourself, so you might consider hiring a professional to build a custom turnkey solution for you to run or even do the entire conversion for you.
First, get the EBCDIC data files and if they exist, the layout files. If you use an FTP program to transfer them, make sure you copy all EBCDIC data files to your PC using "Binary" mode in your FTP program. If you transfer it in ASCII mode (or maybe even in Auto mode), you will corrupt the file and it may not be readable without extensive effort. FileZilla is a widely used free FTP program.
Then, you need to find a program to do the conversion. If you have the simplest type of file (no packed records, no multiple record types, no special fields, etc. you can use the basic vEdit software (32-bit or Pro64 versions). A free trial version of vEdit and our Level-2 EBCDIC Converter is available that works for 30 days.
It's likely your file requires a layout file to do the conversion, but you might not need to handle all possible EBCDIC variations. To help customers optimize their budgets, vEdit comes with three different EBDIC conversion packages (Levels Two, Three and Four) for files of increasing complexity. You can download a free trial version of Level Two that works for 30 days.
You'll need specialized software to do the conversion. vEdit has three EBCDIC conversion packages for increasingly complex EBCDIC files. You also need the layout file layout, preferably a COBOL "copy-book", which describes the location of the different fields in the file.
If you don't have a layout file, we might be able to help you create one from a sample file, as long as your data does not contain multiple types of records (but call us anyway, we might still be able to do it). But that could be a painstaking and expensive process. It's much better to have the layout file.
Only the simplest alpha-numeric only files can be converted like that. It is rare to find those with real-world data. For anything else (like files with packed fields), you'll need specialized software to convert the files. A simple byte-for-byte EBCDIC/ASCII conversion will corrupt the packed fields and recovering them is very difficult if at all possible.
The EBCDIC converter features tables may help you decide which level will work best for you. We're glad to help you figure that out as well. All we need is the file layout specification (like the COBOL "copy-book", data dictionary or other layout specs) and we can usually determine what level is needed. Sometimes we also need to see a sample data file. Contact us now and we'll discuss it with you. If needed, we can arrange to have you send us sample files securely.
If your needs change or for some reason you require a higher-level vEdit EBCDIC Conversion kit, you can upgrade for the price difference within 12 months of your purchase.
We have two options if you don't have the time to learn EBCDIC or just want to accelerate the conversion process:
- We can provide a quote for a ready-to-run custom turnkey solution for you. After we review your project, we create the layout file, convert your sample data, and supply you with the tools to convert the rest of the files yourself by simply running an automated "batch" file.
- If you don't want to run the job at all, you can also hire us do the entire conversion for you.
The conversion process in vEdit is fully automated. The layout file lists the record lengths and the location of the packed-decimal, packed-binary and other special fields. You can start the conversion from a command line, batch file or icon. The automated conversion can even save the file and exit without any further user intervention.
You might even be able to automate the entire process, from the FTP transfer through the conversion.