Skip to content

DataFrameCsvReader should ignore spaces after commas in CSV files #116

Open

Description

@olekscode

The following line of a CSV file:

first_name, last_name, order_date, amount

Will be parser as:

#('first_name' ' last_name' ' order_date' ' amount')

(with each string except the first one starting with a space).

Spaces after commas in CSV files should be ignored

Activity

  1. AtharvaKhare commented on Aug 2, 2019

    @AtharvaKhare
    Contributor

    Shouldn't user just use , (comma space) as separator?

  2. olekscode commented on Jul 26, 2021

    @olekscode
    MemberAuthor

    I think the result should be trimmed from both sides.
    Because sometimes even data inside the same file is inconsistent.

    In most cases, CSV files have data separated by commas, TSV files separate it by tabs.
    But then some people add extra spaces:

    Oleks, 25, true
    

    And others don't:

    Oleks,25,true
    

    It can be even more troublesome when there are tabs and the space can be invisible.

    And then the users will be running into all kinds of problems.
    For example, when ' 25' can not be parsed as a number because there is a space.

    So I think that it's better to trim whitespace characters from left and right when reading from a CSV.
    (but only unless quotes are used! Because if a file contains something like "Oleks", " 25 " then maybe clients want those spaces inside quotes)

  3. added this to the v3.0 milestone on Jul 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions