Nifi ValideCSV Schema example

I am trying to use ValidateCSV processor in Nifi, but I don't know how to define schema. My output(flowfile) is as below:

> PassCountId,CameraId,EventDate,Counter > > 32340,4,2020-10-14 15:26:20.170,4 > > 32341,3,2020-10-14 15:26:51.747,4 > > 32342,3,2020-10-14 15:26:57.907,6

I tried below schema but it didn't work. asked Oct 15, 2020 at 8:55 610 6 6 silver badges 29 29 bronze badges

1 Answer 1

@Tyr here is an example of schema

 "type" : "record", "namespace" : "nifi", "name" : "nifi", "fields" : [ < "name" : "c1" , "type" : ["null", "string"] >, < "name" : "c2" , "type" : ["null", "string"] >, < "name" : "c3" , "type" : ["null", "string"] >] >

As per the documentation you can feed the schema validation functions:

 : [ParseBigDecimal, ParseBool, ParseChar, ParseDate, ParseDouble, ParseInt, ParseLong, Optional, DMinMax, Equals, ForbidSubStr, LMinMax, NotNull, Null, RequireHashCode, RequireSubStr, Strlen, StrMinMax, StrNotNullOrEmpty, StrRegEx, Unique, UniqueHashCode, IsIncludedIn]

My recommendation would be to start with a schema of strings, then experiment with ParseBigDecimal for example for your first column. Work in small testable iterations until you have a fully built schema.