h$h\      !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ Safe-Inferred Implements the relational Table concept. Defines all necessary data types like RTable and RTuple as well as basic relational algebra operations on RTables.(c) Nikos Karagiannidis, 2018BSD3nkarag@gmail.comstablePOSIXNone  #$238 DBFunctorThis exception means that we have tried an Upsert operation where the source t does not have a unique set of Rtuples if grouped by the columns used in the matching condition. This simply means that we cannot determine which of the dublicate ss in the source t will overwrite the target t+, when the matching condition is satisfied. DBFunctor Error message DBFunctorThis exception means that we have tried to do some operation between two RTables, which requires that the structure of the two is the same. e.g., an  Insert Into  TAB RTuples, or a UNION= or toher set operations. By "structure", we mean that the qs and the corresponding data types must match. Essentially what we record in the C must be the same for the two ts DBFunctor3Error message indicating the operation that failed. DBFunctor#Length mismatch between the format  and the input  data RTimestampFormatLengthMismatch = RTimestampFormatLengthMismatch String String deriving(Eq,Show) instance Exception RTimestampFormatLengthMismatchOne (or both) of the input s to function  are empty DBFunctorThis exception is thrown whenever we provide a Timestamp format with not even one valid format pattern  DBFunctorThis exception is thrown whenever we try to access a specific column (i.e., q) of an s! and the column does not exist.  DBFunctorFormat specifier of  style DBFunctor+A map of ColumnName to Format Specification DBFunctor:Basic data type for defining the desired formatting of an s when printing an RTable (see ). DBFunctorFor defining the column ordering (i.e., the SELECT clause in SQL)  DBFunctor*For defining the formating per Column in " style" DBFunctorRTabResult is the result of an RTable operation and is a Writer Monad, that includes the new RTable, as well as the number of RTuples returned by the operation. DBFunctor1Number of RTuples returned by an RTable operation DBFunctorAggregation Function type. An aggregation function receives as input a source column (i.e., a q) of a source t and returns an aggregated value, which is the result of the aggregation on the values of the source column. DBFunctorThis data type represents all possible aggregate operations over an RTable. Examples are : Sum, Count, Average, Min, Max but it can be any other "aggregation". The essential property of an aggregate operation is that it acts on an RTable (or on a group of RTuples - in the case of the RGroupBy operation) and produces a single RTuple.An aggregate operation is applied on a specific column (source column) and the aggregated result will be stored in the target column. It is important to understand that the produced aggregated RTuple is different from the input RTuples. It is a totally new RTuple, that will consist of the aggregated column(s) (and the grouping columns in the case of an RGroupBy). DBFunctor Source column DBFunctor Target column DBFunctor?here we define the aggegate function to be applied on an RTable DBFunctor9The Group By Predicate It defines the condition for two s#s to be included in the same group. DBFunctor*The Upsert Predicate. It defines when two ss should be paired in a merge operation. The matching predicate must be applied on a specific set of matching columns. The source t6 in the Upsert operation must return a unique set of ss, if grouped by this set of matching columns. Otherwise an exception ( ) is thrown." DBFunctor(The Join Predicate. It defines when two ss should be paired.# DBFunctor$A generic binary operation on RTable$ DBFunctor%A generic unary operation on a RTable% DBFunctorA sum type to help the specification of a column ordering (Ascending, or Descending)( DBFunctorDefinition of Relational Algebra operations. These are the valid operations between RTables* DBFunctorUnion + DBFunctor Intersection, DBFunctor Difference- DBFunctor Projection. DBFunctorFilter operation (an B' can be any function of the signature  RTuple -> Bool  so it is much more powerful than a typical SQL filter expression, which is a boolean expression of comparison operators)/ DBFunctorInner Join (any type of join predicate allowed. Any function with a signature of the form:  RTuple -> RTuple -> Bool < is a valid join predicate. I.e., a function which returns  when two RTuples must be paired)0 DBFunctorLeft Outer Join 1 DBFunctorRight Outer Join2 DBFunctor Semi-Join3 DBFunctor Anti-Join 4 DBFunctorPerforms aggregation operations on specific columns and returns a singleton RTable5 DBFunctor/A Group By operation The SQL equivalent is: < SELECT colGrByList, aggList FROM... GROUP BY colGrByList  Note that compared to SQL, we can have a more generic grouping predicate (i.e., when two ss should belong in the same group) than just the equality of values on the common columns between two ss. Also note, that in the case of an aggregation without grouping (equivalent to a single-group group by), then the grouping predicate should be:  _ _ -> True 6 DBFunctorA combination of unary ( s e.g., / (p plist).(f pred) (i.e., RPrj . RFilter)  , in the form of an  RTable -> RTable function.  In this sense we can also include a binary operation (e.g. join), if we partially apply the join to one t, e.g., '(ij jpred rtab) . (p plist) . (f pred) 7 DBFunctorA generic binary (.8 DBFunctor Order the s s of the t acocrding to the specified list of Columns. First column in the input list has the highest priority in the sorting order.< DBFunctorlist of aggregates = DBFunctorthe grouping predicate> DBFunctorthe Group By list of columnsB DBFunctorA Predicate. It defines an arbitrary condition over the columns of an s%. It is used primarily in the filter .+ operation and used in the filter function .C DBFunctor(Basic metadata for a column of an RTupleH DBFunctorBasic Metadata of an s . The s! metadata are accessed through a  q C* structure. I.e., for each column of the s, we access the C structure to get Column-level metadata. This access is achieved by q. However, in order to provide the "impression" of a fixed column order per tuple (see s! definition), we provide another  , the  G q8. So in the follwoing example, if we want to access the H tupmdata ColumnInfo by column order, (assuming that we have N columns) we have to do the following:  (snd tupmdata)!((fst tupmdata)!0) (snd tupmdata)!((fst tupmdata)!1) ... (snd tupmdata)!((fst tupmdata)!(N-1)) 7In the same manner in order to access the column of an s2 (e.g., tup) by column order, we do the following:  tup!((fst tupmdata)!0) tup!((fst tupmdata)!1) ... tup!((fst tupmdata)!(N-1)) I DBFunctorMetadata for an RTableK DBFunctor Name of the tL DBFunctor8Tuple-level metadata other metadataM DBFunctor Primary KeyN DBFunctorList of unique keys i.e., each sublist is a unique key column combinationO DBFunctorBasic data type to represent time. This is a strict data type, meaning whenever we evaluate a value of type O<, there must be also evaluated all the fields it contains.Z DBFunctorDefinition of the Relational Data Type. This is the data type of the values stored in each t. This is a strict data type, meaning whenever we evaluate a value of type Z<, there must be also evaluated all the fields it contains.f DBFunctore.g., "DD/MM/YYYY"i DBFunctor-This is used only for metadata purposes (see C). The actual data type of a value is an RDataType The Text component of Date and Timestamp data constructors is the date format e.g., "DD/MM/YYYY", "DD/MM/YYYY HH24:MI:SS"p DBFunctorDefinition of the Table Nameq DBFunctorDefinition of the Column Namer DBFunctorDefinition of the Name types DBFunctor*Definition of the Relational Tuple. An s is implemented as a  of (q, Z) pairs. This ensures fast access of the column value by column name. Note that this implies that the s CANNOT have more than one columns with the same name (i.e. hashmap key) and more importantly that it DOES NOT have a fixed order of columns, as it is usual in RDBMS implementations. This gives us the freedom to perform column change operations very fast. The only place were we need fixed column order is when we try to load an t from a fixed-column structure such as a CSV file. For this reason, we have embedded the notion of a fixed column-order in the s metadata. See H.t DBFunctor0Definition of the Relational Table entity An t is a "container" of ss.u DBFunctorBasic class to represent a data type that can be turned into an t/. It implements the concept of "tabular data" x DBFunctor Turns an t to a list of ssy DBFunctor(Creates an RTable from a list of RTuplesz DBFunctorTurns an RTuple to a List{ DBFunctorCreate an RTuple from a list| DBFunctorUse this function to compare an RDataType with the Null value because due to Null logic x == Null or x /= Null, will always return False. It returns True if input value is Null} DBFunctorUse this function to compare an RDataType with the Null value because deu to Null logic x == Null or x /= Null, will always return False. It returns True if input value is Not Null~ DBFunctorStandard date format DBFunctor!Get the Column Names of an RTable DBFunctor%Returns the Column Names of an RTuple DBFunctor'Take a column value and return its type DBFunctor#Get the first RTuple from an RTable DBFunctorReturns the value of an RTuple column based on the ColumnName key if the column name is not found, then it returns Nothing DBFunctorReturns the value of an RTuple column based on the ColumnName key if the column name is not found, then it returns a default value DBFunctorgetRTupColValue :: Returns the value of an RTuple column based on the ColumnName key if the column name is not found, then it returns Null. !!!Note that this might be confusing since there might be an existing column name with a Null value!!! DBFunctor?Operator for getting a column value from an RTuple Throws a  8 exception, if this map contains no mapping for the key. DBFunctorSafe Operator for getting a column value from an RTuple if the column name is not found, then it returns Nothing DBFunctorReturns the 1st parameter if this is not Null, otherwise it returns the 2nd.  DBFunctorReturns the value of a specific column (specified by name) if this is not Null. If this value is Null, then it returns the 2nd parameter. If you pass an empty RTuple, then it returns Null. Throws a  8 exception, if this map contains no mapping for the key. DBFunctorIt receives an RTuple and lookups the value at a specfic column name. Then it compares this value with the specified search value. If it is equal to the search value then it returns the specified Return Value. If not, then it returns the specified default Value, if the ignore indicator is not set, otherwise (if the ignore indicator is set) it returns the existing value. If you pass an empty RTuple, then it returns Null. Throws a  8 exception, if this map contains no mapping for the key. DBFunctorIt receives an RTuple and a default value. It returns a new RTuple which is identical to the source one but every Null value in the specified colummn has been replaced by a default value DBFunctorIt receives an RTable and a default value. It returns a new RTable which is identical to the source one but for each RTuple, for the specified column every Null value in every RTuple has been replaced by a default value If you pass an empty RTable, then it returns an empty RTable Throws a  ( exception, if the column does not exist DBFunctorIt receives an RTable, a search value and a default value. It returns a new RTable which is identical to the source one but for each RTuple, for the specified column: if the search value was found then the specified Return Value is returned else the default value is returned (if the ignore indicator is not set), otherwise (if the ignore indicator is set), it returns the existing value for the column for each s. If you pass an empty RTable, then it returns an empty RTable Throws a  ( exception, if the column does not exist DBFunctorstripRText : O(n) Remove leading and trailing white space from a string. If the input RDataType is not an RText, then Null is returned DBFunctorConcatenates two Text  RDataTypes, in all other cases of Z it returns a. DBFunctorHelper function to remove a character around (from both beginning and end) of an (RText t) value DBFunctor Returns an O from an input  and a format .Valid format patterns are: For year: YYYY, e.g., "0001", "2018" For month: MM, e.g., "01", "1", "12" For day: DD , e.g., "01", "1", "31" For hours: HH, HH24 e.g., "00", "23"+ I.e., hours must be specified in 24 format For minutes: MI, e.g., "01", "1", "59" For seconds: SS, e.g., "01", "1", "59"'Example of a typical format string is: "DD/MM/YYYY HH:MI:SS",If no valid format pattern is found then an  exception is thrown DBFunctor Convert an O" value to a Universal Time value () DBFunctor Convert a Universal Time value () to an O value DBFunctor7Search for the first occurence of a substring within a , and return the 1st character position, or  if the substring is not found. DBFunctor7Search for the first occurence of a substring within a 3 string and return the 1st character position, or  if the substring is not found. DBFunctor7Search for the first occurence of a substring within a \3 string and return the 1st character position, or 2 if the substring is not found, or if an non-text Z, is given as input. DBFunctorCreates an RTimestamp data type from an input timestamp format string and a timestamp value represented as a . Valid format patterns are: For year: YYYY, e.g., "0001", "2018" For month: MM, e.g., "01", "1", "12" For day: DD , e.g., "01", "1", "31" For hours: HH, HH24 e.g., "00", "23"+ I.e., hours must be specified in 24 format For minutes: MI, e.g., "01", "1", "59" For seconds: SS, e.g., "01", "1", "59"'Example of a typical format string is: "DD/MM/YYYY HH:MI:SS,If no valid format pattern is found then an  exception is thrown DBFunctorReturn the Text out of an RDataType If a non-text RDataType is given then Nothing is returned. DBFunctor Return an Z from  DBFunctorReturns  only if this is an \ DBFunctor+Standard timestamp format. For example: "DDMMYYYY HH24:MI:SS" DBFunctorrTimeStampToText: converts an RTimestamp value to RText Valid input formats are:1.  "DD/MM/YYYY HH24:MI:SS" 2.  "YYYYMMDD-HH24.MI.SS" 3.  "YYYYMMDD" 4.  "YYYYMM" 5.  "YYYY"  DBFunctorcreateRTableMData : creates RTableMData from input given in the form of a list We assume that the column order of the input list defines the fixed column order of the RTuple. DBFunctortoListColumnName: returns a list of RTuple column names, in the fixed column order of the RTuple. DBFunctortoListColumnInfo: returns a list of RTuple columnInfo, in the fixed column order of the RTuple DBFunctortoListRDataType: returns a list of RDataType values of an RTuple, in the fixed column order of the RTuple DBFunctorDefine equality for two C structures For two column two have "equal structure" they must have the same name and the same type. If one of the two (or both) have an j4, then they are still considered of equal structure.Creates a list of the form [(ColumnInfo, RDataType)] from a list of ColumnInfo and an RTuple. The returned list respects the order of the [ColumnInfo]. It guarantees that RDataTypes will be in the same column order as [ColumnInfo], i.e., the correct RDataType for the correct column DBFunctorcreateRDataType: Get a value of type a and return the corresponding RDataType. The input value data type must be an instance of the Typepable typeclass from Data.Typeable DBFunctor Returns an 5 with a custom aggregation function provided as input DBFunctorThe StrAgg aggregate operation This is known as "string_agg"" in Postgresql and "listagg" in Oracle. It aggregates the values of a text Z" column with a specified delimiter DBFunctorThe Sum aggregate operation DBFunctor>The Count aggregate operation Count aggregation (no distinct) DBFunctor;The CountStar aggregate operation Returns the number of s s in the t (i.e., count(*) in SQL)  DBFunctorThe CountDist aggregate operation Count distinct aggregation (i.e., count(distinct col) in SQL). Returns the distinct number of values for this column. DBFunctorThe Average aggregate operation DBFunctorThe Max aggregate operation DBFunctorThe Min aggregate operation DBFunctorropU operator executes a unary ROperation. A short name for the  function DBFunctorExecute a Unary ROperation DBFunctorropUres operator executes a unary ROperation. A short name for the  function DBFunctor)Execute a Unary ROperation and return an  DBFunctorropB operator executes a binary ROperation. A short name for the  function DBFunctorExecute a Binary ROperation DBFunctorropBres operator executes a binary ROperation. A short name for the  function DBFunctor*Execute a Binary ROperation and return an  DBFunctorTest whether an RTable is empty DBFunctorTest whether an RTuple is empty DBFunctor#emptyRTable: Create an empty RTable DBFunctorCreates an empty RTuple (i.e., one with no column,value mappings) DBFunctor&Creates an RTable with a single RTuple DBFunctorcreateRTuple: Create an Rtuple from a list of column names and values DBFunctorCreates a Null s+ based on a list of input Column Names. A a s is an s( where all column names correspond to a a value (a is a data constructor of Z) DBFunctorReturns  if the input s( is a Null RTuple, otherwise it returns 8 Note that a Null RTuple has all its values equal with a; but it still has columns. This is different from an empty s, which is an s0 withi no columns and no values whatsoever. See . DBFunctorThis is a fold operation on a t that returns an t. It is similar with : 3 foldr' :: (a -> b -> b) -> b -> Vector a -> b  of Vector, which is an O(n) Right fold with a strict accumulator DBFunctorThis is a fold operation on a t that returns an Z value. It is similar with : 3 foldr' :: (a -> b -> b) -> b -> Vector a -> b  of Vector, which is an O(n) Right fold with a strict accumulator DBFunctorThis is a fold operation on t that returns an t. It is similar with : 2 foldl' :: (a -> b -> a) -> a -> Vector b -> a  of Vector, which is an O(n) Left fold with a strict accumulator DBFunctorThis is a fold operation on t that returns an Z value It is similar with : 2 foldl' :: (a -> b -> a) -> a -> Vector b -> a  of Vector, which is an O(n) Left fold with a strict accumulator DBFunctorMap function over an t. DBFunctorO(n) Transform this s& by applying a function to every value DBFunctorO(n) Transform this s& by applying a function to every value DBFunctorCreates an RTuplesRet type DBFunctor6Return the number embedded in the RTuplesRet data type DBFunctorCreates an RTabResult (i.e., a Writer Monad) from a result RTable and the number of RTuples that it returned DBFunctor8Returns the info "stored" in the RTabResult Writer Monad DBFunctorReturns the "log message" in the RTabResult Writer Monad, which is the number of returned RTuples DBFunctorremoveColumn : removes a column from an RTable. The column is specified by ColumnName. If this ColumnName does not exist in the RTuple of the input RTable then nothing is happened, the RTuple remains intact. DBFunctor%addColumn: adds a column to an RTable DBFunctor7Filter (i.e. selection operator). A short name for the  runRFilter function DBFunctorExecutes an RFilter operation DBFunctor1RTable Projection operator. A short name for the  function DBFunctorImplements RTable projection operation. If a column name does not exist, then an empty RTable is returned. DBFunctorImplements RTable projection operation. If a column name does not exist, then the returned RTable includes this column with a Null value. This projection implementation allows missed hits. DBFunctorreturns the N first ss of an t DBFunctort* anti-join operator. A short name for the  function DBFunctorImplements the anti-Join operation between two RTables (any type of join predicate is allowed) It returns the ss from the left t that DONT match with the right t. DBFunctort* semi-join operator. A short name for the  function DBFunctorImplements the semi-Join operation between two RTables (any type of join predicate is allowed) It returns the ss from the left t that match with the right t. Note that if an s from the left t matches more than one ss from the right t4 the semi join operation will return only a single s.  DBFunctort+ Inner Join Operator. A short name for the  function DBFunctorImplements an Inner Join operation between two RTables (any type of join predicate is allowed) This Inner Join implementation follows Oracle DB's convention for common column names. When we have two tuples t1 and t2 with a common column name (lets say "Common"), then the resulting tuple after a join will be "Common", "Common_1", so a "_1" suffix is appended. The tuple from the left table by convention retains the original column name. So "Column_1" is the column from the right table. If "Column_1" already exists, then "Column_2" is used. DBFunctorJoins two RTuples into one. In this join we follow Oracle DB's convention when joining two tuples with some common column names. When we have two tuples t1 and t2 with a common column name (lets say Common2), then the resulitng tuple after a join will be Common, Common_1, so a "_1" suffix is appended. The tuple from the left table by convention retains the original column name. So Column_1) is the column from the right table. If Column_1 already exists, then Column_2 is used. DBFunctor6RTable Left Outer Join Operator. A short name for the  function DBFunctorImplements a Left Outer Join operation between two RTables (any type of join predicate is allowed), i.e., the rows of the left RTable will be preserved. Note that when dublicate keys encountered that is, since the underlying structure for an RTuple is a Data.HashMap.Strict, only one value per key is allowed. So in the context of joining two RTuples the value of the left RTuple on the common key will be prefered.Implements a Left Outer Join operation between two RTables (any type of join predicate is allowed), i.e., the rows of the left RTable will be preserved. A Left Join : + tabLeft LEFT JOIN tabRight ON joinPred  where tabLeft is the preserving table can be defined as: the Union between the following two RTables:The result of the inner join: tabLeft INNER JOIN tabRight ON joinPredThe rows from the preserving table (tabLeft) that DONT satisfy the join condition, enhanced with the columns of tabRight returning Null values.The common columns will appear from both tables but only the left table column's will retain their original name.  DBFunctorReceives two lists of q!s and returns the unique list of qs after concatenating the two and removing the names from the second one that are a prefix of the first one. This function is intended to dedublicate common columns after a join (see ij ), where ColA# for example, will also appear as ColA_1-. This function DOES NOT dedublicate columns ColA and ColAsomeSuffix, only cases like this one  ColName_Num8 (e.g., ColName_1, ColName_2, etc.) Here is an example:getUniqueColumnNames ["ColA","ColB"] ["ColC","ColA", "ColA_1", "ColA_2", "ColA_A", "ColA_hello", "ColAhello"]8["ColA","ColB","ColC","ColA_A","ColA_hello","ColAhello"] DBFunctor7RTable Right Outer Join Operator. A short name for the  function DBFunctorImplements a Right Outer Join operation between two RTables (any type of join predicate is allowed), i.e., the rows of the right RTable will be preserved. A Right Join : , tabLeft RIGHT JOIN tabRight ON joinPred  where tabRight is the preserving table can be defined as: the Union between the following two RTables:The result of the inner join: tabLeft INNER JOIN tabRight ON joinPredThe rows from the preserving table (tabRight) that DONT satisfy the join condition, enhanced with the columns of tabLeft returning Null values.The common columns will appear from both tables but only the right table column's will retain their original name.  DBFunctorImplements a Right Outer Join operation between two RTables (any type of join predicate is allowed) i.e., the rows of the right RTable will be preserved. Note that when dublicate keys encountered that is, since the underlying structure for an RTuple is a Data.HashMap.Strict, only one value per key is allowed. So in the context of joining two RTuples the value of the right RTuple on the common key will be prefered.6RTable Full Outer Join Operator. A short name for the  function DBFunctorImplements a Full Outer Join operation between two RTables (any type of join predicate is allowed) A full outer join is the union of the left and right outer joins respectively. The common columns will appear from both tables but only the left table column's will retain their original name (just by convention). DBFunctor,RTable Union Operator. A short name for the  function DBFunctorImplements the union of two RTables as a union of two lists (see ). Duplicates, and elements of the first list, are removed from the the second list, but if the first list contains duplicates, so will the result:Implements the union of two RTables. Note that dublicate s elimination takes places. DBFunctorImplements the union-all of two RTables. I.e., a union without dublicate s elimination. Runs in O(m+n). DBFunctor3RTable Intersection Operator. A short name for the  function DBFunctor+Implements the intersection of two RTables  DBFunctor1RTable Difference Operator. A short name for the  function DBFunctorImplements the set Difference of two RTables as the diff of two lists (see ). DBFunctor+Aggregation Operator. A short name for the  function DBFunctorImplements the aggregation operation on an RTable It aggregates the specific columns in each AggOperation and returns a singleton RTable i.e., an RTable with a single RTuple that includes only the agg columns and their aggregated value. DBFunctor(Order By Operator. A short name for the  function DBFunctorImplements the ORDER BY operation. First column in the input list has the highest priority in the sorting order We treat Null as the maximum value (anything compared to Null is smaller). This way Nulls are send at the end (i.e., "Nulls Last" in SQL parlance). This is for Asc ordering. For Desc ordering, we have the opposite. Nulls go first and so anything compared to Null is greater. @ SQL example with q as (select case when level < 4 then level else NULL end c1 -- , level c2 from dual connect by level < 7 ) select * from q order by c1>C1 ---- 1 2 3 Null Null Nullwith q as (select case when level < 4 then level else NULL end c1 -- , level c2 from dual connect by level < 7 ) select * from q order by c1 desc DBFunctor(Group By Operator. A short name for the  function DBFunctor'Implement a grouping operation over an t. No aggregation takes place. It returns the individual groups as separate t)s in a list. In total the initial set of ss is retained. If an empty t; is provided as input, then a ["empty RTable"] is returned. DBFunctorConcatenates a list of t2s to a single RTable. Essentially, it unions (see ) all ts of the list. DBFunctor'Implement a grouping operation over an t*. No aggregation takes place. The output t has exactly the same ss, as the input, but these are grouped based on the input grouping predicate. If an empty t% is provided as input, then an empty t is returned. DBFunctor*Implements the GROUP BY operation over an t.  DBFunctorA short name for the  function DBFunctorrunCombinedROp: A Higher Order function that accepts as input a combination of unary ROperations e.g., (p plist).(f pred) expressed in the form of a function (RTable -> Rtable) and applies this function to the input RTable. In this sense we can also include a binary operation (e.g. join), if we partially apply the join to one RTable e.g., (ij jpred rtab) . (p plist) . (f pred) DBFunctorO(n) append an RTuple to an RTable Please note that this is an  immutable implementation of an t insert. This simply means that the insert operation returns a new t# and does not affect the original t. DBFunctorO(n) prepend an RTuple to an RTable Please note that this is an  immutable implementation of an t insert. This simply means that the insert operation returns a new t# and does not affect the original t. DBFunctor Insert an t to an existing t. This is equivalent to an INSERT INTO SELECT+ caluse in SQL. We want to insert into an t the results of a "subquery", which in our case is materialized via the input t. Please note that this is an  immutable implementation of an t insert. This simply means that the insert operation returns a new t# and does not affect the original t(. Also note that the source and target ts should have the same structure. By "structure", we mean that the qs and the corresponding data types must match. Essentially what we record in the C must be the same for the two ts. Otherwise a  exception will be thrown. DBFunctorUpsert (Update+Insert, aka Merge) Operation. We provide a source t and a matching condition ( ) to the ss of the target t. An s from the target t might match to a single only s in the source t;, or not match at all. If it is matched to more than one ss then an exception () is thrown. When an s from the target t is matched to a source s/, then the corresponding columns of the target s9 are updated with the new values provided in the source s". This takes place for the target s/s that match but also that satisfy the input B1. Thus we can restrict further with a filter the ss of the target t8 where the update will take place. Finally, the source s#s that did not match to the target t(, are inserted (appended) to the target tPlease note that this is an  immutable implementation of an t upsert. This simply means that the upsert operation returns a new t# and does not affect the original t5. Moreover, if we have multiple threads updating an t?, due to immutability, each thread "sees" its own copy of the t3 and thus there is no need for locking the updated s s, as happens in a common RDBMS.%Also note that the source and target ts should have the same structure. By "structure", we mean that the qs and the corresponding data types must match. Essentially what we record in the C must be the same for the two ts. Otherwise a  exception will be thrown.  An Example: Source RTable: src = Id | Msg | Other ----|---------------|------- 1 | "hello2" |"a" 2 | "world2" |"a" 3 | "new" |"a" Target RTable: trg = Id | Msg | Other ----|---------------|------- 1 | "hello1" |"b" 2 | "world1" |"b" 4 | "old" |"b" 5 | "hello" |"b" >>> upsertRTab src RUpsertPredicate {matchCols = ["Id"], matchPred = \t1 t2 -> t1 "Id" == t2 "Id" } ["Msg"] (\t -> let msg = case toText (t "Msg") of Just t -> t Nothing -> pack "" in (take 5 msg) == (pack "hello") ) -- Msg like "hello%" trg Result RTable: rslt = Id | Msg | Other ----|---------------|------- 1 | "hello2" |"b" (Note that only column "Msg" has been overwritten, as per the 3rd argument) 2 | "world1" |"b" 3 | "new" |"a" 4 | "old" |"b" 5 | "hello" |"b"  DBFunctor$Compares the structure of the input ts and returns : if these are the same. By "structure", we mean that the qs and the corresponding data types must match. Essentially what we record in the C must be the same for the two ts. Note that in the case of two columns having the same name but one of the two (or both) have a F equal to j, then this function assumes that they are the same (i.e., equal Cs). DBFunctor$Compares the structure of the input ss and returns : if these are the same. By "structure", we mean that the qs and the corresponding data types must match. Essentially what we record in the C must be the same for the two ss DBFunctorDelete s s from an t based on an B. Please note that this is an  immutable implementation of an t update. This simply means that the delete operation returns a new t. So, the original t remains unchanged and no deletion in-place takes place whatsoever. Moreover, if we have multiple threads deleting an t?, due to immutability, each thread "sees" its own copy of the t3 and thus there is no need for locking the deleted s s, as happens in a common RDBMS. DBFunctorUpdate an RTable. The input includes a list of (ColumnName, new Value) pairs. Also a filter predicate is specified, in order to restrict the update only to those s;s that fulfill the predicate. Please note that this is an  immutable implementation of an t update. This simply means that the update operation returns a new t that includes all the ss of the original t, both the ones that have been updated and the others that have not. So, the original t remains unchanged and no update in-place takes place whatsoever. Moreover, if we have multiple threads updating an t?, due to immutability, each thread "sees" its own copy of the t3 and thus there is no need for locking the updated s s, as happens in a common RDBMS. DBFunctorUpdate an RTuple at a specific column specified by name with a value. If the q exists, then the value is updated with the input value. If the q does not exist, then a   exception is thrown. DBFunctorUpsert (update/insert) an RTuple at a specific column specified by name with a value If the cname key is not found then the (columnName, value) pair is inserted. If it exists then the value is updated with the input value. DBFunctor/Generates a default Column Format Specification DBFunctor'Generates a Column Format Specification DBFunctor+Generate an RTupleFormat data type instance DBFunctorGenerate a default RTupleFormat data type instance. In this case the returned column order (Select list), will be unspecified and dependant only by the underlying structure of the s () DBFunctorSafe  printRfTable alternative that returns an , so as to give the ability to handle exceptions gracefully, during the evaluation of the input RTable. Example: do p <- (eitherPrintfRTable printfRTable myFormat myRTab) :: IO (Either SomeException ()) case p of Left exc -> putStrLn $ "There was an error in the Julius evaluation: " ++ (show exc) Right _ -> return ()  DBFunctorprints an RTable with an RTuple format specification. It can be used instead of + when one of the following two is required:a) When we want to specify the order that the columns will be printed on screenb) When we want to specify the formatting of the values by using a -like   DBFunctorSafe  alternative that returns an , so as to give the ability to handle exceptions gracefully, during the evaluation of the input RTable. Example: do p <- (eitherPrintRTable printRTable myRTab) :: IO (Either SomeException ()) case p of Left exc -> putStrLn $ "There was an error in the Julius evaluation: " ++ (show exc) Right _ -> return ()  DBFunctor.printRTable : Print the input RTable on screen DBFunctorIn order to be able to force full evaluation up to Normal Form (NF) DBFunctor-In order to be able to use (/) with RDataType DBFunctorWe need to explicitly specify equation of RDataType due to SQL NULL logic (i.e., anything compared to NULL returns false):  Null == _ = False, _ == Null = False, Null /= _ = False, _ /= Null = False.  IMPORTANT NOTE: Of course this means that anywhere in your code where you have something like this:  x == Null or x /= Null,  will always return False and thus it is futile to do this comparison. You have to use the is | function instead. DBFunctorIn order to be able to force full evaluation up to Normal Form (NF) list of column names to be included in the final result RTable DBFunctor>list of column names to be included in the final result RTable DBFunctor number of N s s to return DBFunctorinput t DBFunctoroutput t DBFunctorInput Aggregate Operations DBFunctor Input RTable DBFunctorOutput singleton RTable DBFunctorInput ordering specification DBFunctor Input RTable DBFunctor Output RTable DBFunctor3Grouping predicate, in order to form the groups of ss (it defines when two s's should be included in the same group) DBFunctorList of grouping column names (GROUP BY clause in SQL) We assume that all RTuples in the same group have the same value in these columns DBFunctorinput t DBFunctoroutput list of t's where each one corresponds to a group DBFunctor3Grouping predicate, in order to form the groups of ss (it defines when two s's should be included in the same group) DBFunctorList of grouping column names (GROUP BY clause in SQL) We assume that all s8s in the same group have the same value in these columns DBFunctorinput t DBFunctoroutput t DBFunctorGrouping predicate, in order to form the groups of RTuples (it defines when two RTuples should be included in the same group) DBFunctor.Aggregations to be applied on specific columns DBFunctorList of grouping column names (GROUP BY clause in SQL) We assume that all RTuples in the same group have the same value in these columns DBFunctor input RTable DBFunctor output RTable DBFunctorinput combined RTable operation DBFunctor7input RTable that the input function will be applied to DBFunctor output RTable DBFunctorSource t to be inserted DBFunctorTarget t DBFunctor Final Result DBFunctorSource t!, i.e., the equivalent to an SQL USING subclause  DBFunctorThe s+ matching predicate for the merge operation DBFunctorList of column names to be updated with the corresponding new values coming from the source ss that match with the target ss based on the   DBFunctor#A filter that specifies the target ss to be updated DBFunctor The target t DBFunctor Final Result DBFunctorPredicate specifying the Rtuples that must be deleted DBFunctort" that the deletion will be applied DBFunctorResult t DBFunctorList of column names to be updated with the corresponding new values DBFunctorAn RTuple -> Bool function that specifies the RTuples to be updated DBFunctor Input RTable DBFunctor Output RTable DBFunctor$key where the update will take place DBFunctor new value DBFunctor input RTuple DBFunctor output RTuple DBFunctor$key where the upsert will take place DBFunctor new value DBFunctor input RTuple DBFunctor output RTuple DBFunctorColumn Select list  DBFunctorColumn Format Map DBFunctorOutput   !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghiokjlmnpqrstuvwxyz{|}~tsZ[\]^_`abcdefghOPQRSTUVIJKLMNHCDEFGrqpiokjlmnuvw()*+,-./0123456789:;<=>?@A$#B" !WXY~|}xzy{    %&''Implements ETL operations over RTables.(c) Nikos Karagiannidis, 2018BSD3nkarag@gmail.comstablePOSIXNone  DBFunctorETLmapping : it is the equivalent of a mapping in an ETL tool and consists of a series of ETLOperations that are applied, one-by-one, to some initial input RTable, but if binary ETLOperations are included in the ETLMapping, then there will be more than one input RTables that the ETLOperations of the ETLMapping will be applied to. When we apply (i.e., run) an ETLOperation of the ETLMapping we get a new RTable, which is then inputed to the next ETLOperation, until we finally run all ETLOperations. The purpose of the execution of an ETLMapping is to produce a single new RTable as the result of the execution of all the ETLOperations of the ETLMapping. In terms of database operations an ETLMapping is the equivalent of an CREATE AS SELECT (CTAS) operation in an RDBMS. This means that anything that can be done in the SELECT part (i.e., column projection, row filtering, grouping and join operations, etc.) in order to produce a new table, can be included in an ETLMapping.?An ETLMapping is executed with the etl (runETLmapping) operatorImplementation: An ETLMapping is implemented as a binary tree where the node represents the ETLOperation to be executed and the left branch is another ETLMapping, while the right branch is an RTable (that might be empty in the case of a Unary ETLOperation). Execution proceeds from bottom-left to top-right. This is similar in concept to a left-deep join tree. In a Left-Deep ETLOperation tree the "pipe" of ETLOperations comes from the left branches always. The leaf node is always an ETLMapping with an ETLMapEmpty in the left branch and an RTable in the right branch (the initial RTable inputed to the ETLMapping). In this way, the result of the execution of each ETLOperation (which is an RTable) is passed on to the next ETLOperation. Here is an example:  A Left-Deep ETLOperation Tree final RTable result / etlOp3 / etlOp2 rtab2 / A leaf-node --> etlOp1 emptyRTab / ETLMapEmpty rtab1 You see that always on the left branch we have an ETLMapping data type (i.e., a left-deep ETLOperation tree). So how do we implement the following case?  final RTable result / A leaf-node --> etlOp1 / rtab1 rtab2 The answer is that we "model" the left RTable (rtab1 in our example) as an ETLMapping of the form:  ETLMapLD { etlOp = ETLcOp{cmap = ColMapEmpty}, tabL = ETLMapEmpty, tabR = rtab1 } So we embed the rtab1 in a ETLMapping, which is a leaf (i.e., it has an empty prevMap), the rtab1 is in the right branch (tabR) and the ETLOperation is the EmptyColMapping, which returns its input RTable when executed. We can use function  for this job. So it becomes  A leaf-node --> etlOp1 / rtabToETLMapping rtab1 rtab2 >In this manner, a leaf-node can also be implemented like this:  final RTable result / etlOp3 / etlOp2 rtab2 / A leaf-node --> etlOp1 emptyRTab / rtabToETLMapping rtab1 emptyRTable  DBFunctor an empty node DBFunctora Left-Deep node DBFunctora Right-Deep node DBFunctora Balanced node DBFunctor#the ETLOperation to be executed  DBFunctorthe left-branch corresponding to the previous ETLOperation, which is input to this one. DBFunctorthe right branch corresponds to another RTable (for binary ETL operations). If this is a Unary ETLOperation then this field must be an empty RTable. DBFunctorthe left-branch corresponds to another RTable (for binary ETL operations). If this is a Unary ETLOperation then this field must be an empty RTable.  DBFunctorthe right branch corresponding to the previous ETLOperation, which is input to this one. DBFunctorthe left-branch corresponding to the previous ETLOperation, which is input to this one. If this is a Unary ETLOperation then this field might be an empty ETLMapping.  DBFunctorthe right branch corresponding corresponding to the previous ETLOperation, which is input to this one. -- If this is a Unary ETLOperation then this field might be an empty ETLMapping. DBFunctor7An ETL operation applied to an RTable can be either an ( (a relational agebra operation like join, filter etc.) defined in  RTable.Core module, or an  applied to an t DBFunctorA Column Transformation function data type. It is used in order to define an arbitrary column-level transformation (i.e., from a list of N input Column-Values we produce a list of M derived (output) Column-Values). A Column value is represented with the Z. DBFunctorThis is the basic data type to define the column-to-column mapping from a source t to a target t. Essentially, an 3 represents the column-level transformations of an s that will yield a target s. A mapping is simply a triple of the form ( Source-Column(s), Target-Column(s), Transformation, RTuple-Filter), where we define the source columns over which a transformation (i.e. a function) will be applied in order to yield the target columns. Also, an B0 (i.e. a filter) might be applied on the source s. Remember that an s is essentially a mapping between a key (the Column Name) and a value (the Z value). So the various  data constructors below simply describe the possible modifications of an s! orginating from its own columns.So, we can have the following mapping types: a) single-source column to single-target column mapping (1 to 1), the source column will be removed or not based on the 4 flag (dublicate column names are not allowed in an s) b) multiple-source columns to single-target column mapping (N to 1), The N columns will be merged to the single target column based on the transformation. The N columns will be removed from the RTuple or not based on the 4 flag (dublicate column names are not allowed in an s) c) single-source column to multiple-target columns mapping (1 to M) the source column will be "expanded" to M target columns based ont he transformation. the source column will be removed or not based on the 4 flag (dublicate column names are not allowed in an s) d) multiple-source column to multiple target columns mapping (N to M) The N source columns will be mapped to M target columns based on the transformation. The N columns will be removed from the RTuple or not based on the 2 flag (dublicate column names are not allow in an s)+Some examples of mapping are the following:  ( Start_Date, No,  StartDate, t -> True) -- copy the source value to target and dont remove the source column, so the target RTuple will have both columns  Start_Date and  StartDate -- with the exactly the same value) ([Amount, Discount], Yes,  FinalAmount, ([a, d] -> a * d) ) --  FinalAmount is a derived column based on a function applied to the two source columns. -- In the final RTuple we remove the two source columns. An  can be applied with the  (runColMapping) operator DBFunctor>single-source column to single-target column mapping (1 to 1). DBFunctormultiple-source columns to single-target column mapping (N to 1) DBFunctorsingle-source column to multiple-target columns mapping (1 to N) DBFunctormultiple-source column to multiple target columns mapping (N to M)  DBFunctorConstructs an RColMapping. This is the suggested method for creating a column mapping and not by calling the data constructors directly. DBFunctorrunCM operator executes an RColMapping If a target-column has the same name with a source-column and a DontRemoveSrc (i.e., removeSrcCol == No) has been specified, then the (target-column, target-value) key-value pair, overwrites the corresponding (source-column, source-value) key-value pair DBFunctorexecutes a Unary ETL Operation DBFunctorexecutes a Binary ETL Operation DBFunctor etlOp1 emptyRTab / ETLMapEmpty rtab1  DBFunctor1creates a Binary operation leaf node of the form:  A leaf-node --> etlOp1 / rtabToETLMapping rtab1 rtab2  DBFunctorConnects an ETL Mapping to a left-deep ETL Mapping tree, of the form  A Left-Deep ETLOperation Tree final RTable result / etlOp3 / etlOp2 rtab2 / A leaf-node --> etlOp1 emptyRTab / ETLMapEmpty rtab1 Example:  -- connect a Unary ETL mapping (etlOp2) etlOp2 / etlOp1 emptyRTab => connectETLMapLD etlOp2 emptyRTable prevMap -- connect a Binary ETL Mapping (etlOp3) etlOp3 / etlOp2 rtab2 => connectETLMapLD etlOp3 rtab2 prevMap Note that the right branch (RTable) appears first in the list of input arguments of this function and the left branch (ETLMapping) appears second. This is strange, and one could thought that it is a mistake (i.e., the left branch should appear first and the right branch second) since we are reading from left to right. However this was a deliberate choice, so that we leave the left branch (which is the connection point with the previous ETLMapping) as the last argument, and thus we can partially apply the argumenets and get a new function with input parameter only the previous mapping. This is very helpfull in function composition DBFunctorThis operator executes an  DBFunctorThis operator executes an  and returns the  Writer Monad that embedds apart from the resulting RTable, also the number of s s returned DBFunctor Model an t as an + which when executed will return the input t DBFunctorList of source column names DBFunctorList of target column names DBFunctorColumn Transformation function DBFunctorRemove source column option DBFunctorFiltering predicate DBFunctorOutput Column Mapping DBFunctor!ETL operation of this ETL mapping DBFunctor input RTable DBFunctoroutput ETLMapping DBFunctor!ETL operation of this ETL mapping DBFunctor input RTable1 DBFunctor input RTable2 DBFunctoroutput ETLMapping DBFunctor!ETL operation of this ETL Mapping DBFunctorRight RTable (right branch) (if this is a Unary ETL mapping this should be an emptyRTable)  DBFunctor*Previous ETL mapping (left branch)  DBFunctor8New ETL Mapping, which has added at the end the new node DBFunctorinput ETLMapping DBFunctoroutput RTabResult// (EtlR $ ROpStart :.(Upsert $ MergeInto (Tab trgTab) $ Using (TabSrc srcTab) $ MergeOn (RUpsertPredicate ["Id"] (\t1 t2 -> t1 "Id" == t2 "Id")) $ -- merge condition: srcTab.Id == trgTab.Id WhenMatchedThen $ UpdateCols ["Msg"] $ FilterBy (\t -> let msg = case toText (t "Msg") of Just t -> t Nothing -> pack "" in (take 5 msg) == (pack "hello") ) -- Msg like "hello%" ) ) Result RTable: Id | Msg | Other ----|---------------|------- 1 | "updated" |"b" -- Updated RTuple. Note that only column "Msg" has been overwritten, as per the UpdateCols subclause 2 | "world1" |"b" -- Not affected due to FilterBy predicate 3 | "inserted" |"a" -- Inserted RTuple 4 | "old" |"b" -- Not affected due to MergeOn condition 5 | "hello" |"b" -- Not affected due to MergeOn condition  DBFunctor#A Relational Operation Expression () is a sequence of one or more Relational Algebra Operations applied on a input t6. It is a sub-expression within a Julius Expression () and we use it whenever we want to apply relational algebra operations on an RTable (which might be the result of previous operations in a Julius Expression). A Julius Expression (&) can contain an arbitrary number of 's. The relational operation connector " is left associative because in a  operations are evaluated from left to right (or top to bottom). DBFunctor Predicate for Deletion Operation DBFunctorAn s predicate clause. DBFunctorIndicator of whether the source column(s) in a Column Mapping will be removed or not (used in ) If a target-column has the same name with a source-column and a  has been specified, then the (target-column, target-value) key-value pair, overwrites the corresponding (source-column, source-value) key-value pair. DBFunctor Defines the t/ that the current operation will be applied to. DBFunctorDefines the column transformation function of a Column Mapping Expression ( ), the input t: that this transformation will take place, an indicator () of whether the Source Columns will be removed or not in the new t that will be created after the Column Mapping is executed and finally, an s filter predicate () that defines the subset of ss that this Column Mapping will be applied to. If it must be applied to all s!s, then for the last parameter (%), we can just provide the following B: FilterBy (\_ -> True)  DBFunctor;Defines the Target Columns of a Column Mapping Expression (*) and the column transformation function (). DBFunctorA Column Mapping () is the main ETL/ELT construct for defining a column-level transformation. Essentially with a Column Mapping we can create one or more new (derived) column(s) (Target Columns2), based on an arbitrary transformation function (6) with input parameters any of the existing columns (Source Columns ). So a  is either empty, or it defines the source columns, the target columns and the transformation from source to target. Notes: * If a target-column has the same name with a source-column and a , or a  has been specified, then the (target-column, target-value) key-value pair, overwrites the corresponding (source-column, source-value) key-value pair * The returned t will include only the s5s that satisfy the filter predicate specified in the  clause. DBFunctor4A named intermediate result in a Julius expression (), which we can access via the  function. DBFunctorThe name of an intermediate result, used as a key for accessing this result via the  function. DBFunctorAn ETL Operation Expression is either a Column Mapping Expression ()), or a Relational Operation Expression () DBFunctorAn ETL Mapping Expression is a "Julius Expression". It is a sequence of individual ETL Operation Expressions. Each such ETL Operation "acts" on some input t) and produces a new "transformed" output t. The ETL Mapping connector  (as well as the - connector) is left associative because in a  operations are evaluated from left to right (or top to bottom) A Named ETL Operation Expression () is just an ETL Operation with a name, so as to be able to reference this specific step in the chain of ETL Operations. It is actually a named intermediate result, which we can reference and use in other parts of our Julius expression DBFunctorReturns a prefix of an ETLMappingExpr that matches a named intermediate result. For example, below we show a Julius expression where we define an intermediate named result called "myResult". This result, is used at a later stage in this Julius expression, with the use of the function takeNamedResult.  etlXpression = EtlMapStart :-> (EtlC $ ...) :=> NamedResult "myResult" (EtlR $ ...) :-> (EtlR $ ... ) :-> (EtlR $ ROpStart :. (Minus (TabL $ juliusToRTable $ takeNamedResult "myResult" etlXpression -- THIS IS THE POINT WHERE WE USE THE NAMED RESULT! ) (Previous)) ) In the above Julius expression (etlXpresion) the "myResult" named result equals to the prefix of the etlXpresion, up to the operation (included) with the named result "myResult".  takeNamedResult "myResult" etlXpression == EtlMapStart :-> (EtlC $ ...) :=> NamedResult "myResult" (EtlR $ ...) Note that the julius expression is scanned from right to left and thus it will return the longest prefix expression that matches the input name DBFunctor9Evaluates (parses) the Julius exrpession and produces an . The  is an internal representation of the Julius expression and one needs to combine it with the > function, in order to evaluate the Julius expression into an t.. This can be achieved directly with function  DBFunctorPure code to evaluate the "ETL-logic" of a Julius expression and generate the corresponding target RTable./The evaluation of a Julius expression (i.e., a ) to an RTable is strict. It evaluates fully to Normal Form (NF) as opposed to a lazy evaluation (i.e., only during IO), or evaluation to a WHNF. This is for efficiency reasons (e.g., avoid space leaks and excessive memory usage). It also has the impact that exceptions will be thrown at the same line of code that 2 is called. Thus one should wrap this call with a  handler, or use , or 2, if one wants to handle the exception gracefully.Example: do catch (printRTable $ juliusToRTable $ ) (\e -> putStrLn $ "There was an error in the Julius evaluation: " ++ (show (e::SomeException)) )  Or, similarly do p <- (eitherPrintRTable printRTable $ juliusToRTable $ ) :: IO (Either SomeException ()) case p of Left exc -> putStrLn $ "There was an error in the Julius evaluation: " ++ (show exc) Right _ -> return ()  DBFunctorEvaluate a Julius expression within the IO Monad. I.e., Effectful code to evaluate the "ETL-logic" of a Julius expression and generate the corresponding target RTable./The evaluation of a Julius expression (i.e., a ) to an RTable is strict. It evaluates fully to Normal Form (NF) as opposed to a lazy evaluation (i.e., only during IO), or evaluation to a WHNF. This is for efficiency reasons (e.g., avoid space leaks and excessive memory usage). It also has the impact that exceptions will be thrown at the same line of code that 2 is called. Thus one should wrap this call with a  handler, or use 2, if he wants to handle the exception gracefully.Example: do result <- catch (runJulius $ ) (e -> do putStrLn $ "there was an error in Julius evaluation: " ++ (show (e::SomeException)) return emptyRTable )  DBFunctorEvaluate a Julius expression and return the corresponding target t or an exception. One can define custom exceptions to be thrown within a Julius expression. This function will catch any exceptions that are instances of the  type class./The evaluation of a Julius expression (i.e., a ) to an t is strict. It evaluates fully to Normal Form (NF) as opposed to a lazy evaluation (i.e., only during IO), or evaluation to a WHNF. This is for efficiency reasons (e.g., avoid space leaks and excessive memory usage). Example: do res <- (eitherRunJulius $ ) :: IO (Either SomeException RTable) resultRTab <- case res of Right t -> return t Left exc -> do putStrLn $ "there was an error in Julius evaluation: " ++ (show exc) return emptyRTable  DBFunctorReceives an input Julius expression, evaluates it to an ETL Mapping (+) and executes it, in order to return an  containing an t storing the result of the ETL Mapping, as well as the number of s s returned  DBFunctor?Evaluate a Julius expression within the IO Monad and return an . DBFunctorEvaluate a Julius expression within the IO Monad and return either an 9, or an exception, in case of an error during evaluation. DBFunctorGeneric ETL execution function. It receives a list of input (aka "source") ts and an ETL function that produces a list of output (aka "target") ts. The ETL function should embed all the "transformation-logic" from the source ts to the target ts. DBFunctorGeneric ETL execution function that returns either the target list of ts, or an exception in case of a problem during the ETL code execution. It receives a list of input (aka "source") ts and an ETL function that produces a list of output (aka "target") ts. The ETL function should embed all the "transformation-logic" from the source ts to the target ts. DBFunctor Returns an - that adds a surrogate key (SK) column to an t and fills each row with a SK value. This function is only exposed for backward compatibility reasons. The recommended function to use instead is ?, which can be embedded directly into a Julius expression as a $. DBFunctor Returns an $ (t -> t.) that adds a surrogate key (SK) column to an t and fills each row with a SK value. It primarily is intended to be used within a Julius expression. For example: 9 GenUnaryOp (On Tab rtab1) $ ByUnaryOp (addSurrogateKeyJ TxSK 0)  DBFunctor Returns an  that Appends an t to a target t This function is only exposed for backward compatibility reasons. The recommended function to use instead is ?, which can be embedded directly into a Julius expression as a #. DBFunctor Returns a # (t -> t -> t) that Appends an t to a target t. It is primarily intended to be used within a Julius expression. For example:  GenBinaryOp (TabL rtab1) (Tab $ rtab2) $ ByBinaryOp appendRTableJ  DBFunctor#the name of the intermediate result DBFunctorinput ETLMapping Expression DBFunctoroutput ETLMapping Expression DBFunctorThe name of the surrogate key column -> Integer -- ^ The initial value of the Surrogate Key will be the value of this parameter  DBFunctorThe initial value of the Surrogate Key will be the value of this parameter  DBFunctorOutput ETL operation which encapsulates the add surrogate key column mapping DBFunctor$The name of the surrogate key column DBFunctorThe initial value of the Surrogate Key will be the value of this parameter  DBFunctor Input RTable DBFunctor Output RTable DBFunctorOutput ETL Operation DBFunctor Target RTable DBFunctor Input RTable DBFunctorOutput RTable    !"#$%&'(876543210/.-,+)*A@?>=<;9:BCDEFGHIJNMKLOPVUTSQRWXYZa`_^][\hgfedbcinmljokpqrstuvwxyz{|}~655 Implements t3 over CSV (TSV, or any other delimiter) files logic(c) Nikos Karagiannidis, 2018BSD3nkarag@gmail.comstablePOSIXNone #$>\? DBFunctor/This exception signifies an error in parsing a   to an Z value DBFunctor?@ABCDEFGHHIJKLMMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~      (DBFunctor-0.1.2.0-91MwF3Y02wXHpQetuSHbNf RTable.CoreEtl.Internal.Core Etl.JuliusRTable.Data.CSVPaths_DBFunctorDataListData.CsvHeaderbase Control.Monad<=<GHC.Base.UniquenessViolationInUpsertConflictingRTableStructuresEmptyInputStringsInToRTimestampUnsupportedTimeStampFormatColumnDoesNotExistFormatSpecifier DefaultFormatFormat ColFormatMap RTupleFormat colSelectList colFormatMap RTabResult RTuplesRet Delimiter AggFunction RAggOperation sourceCol targetColaggFuncRGroupPredicateRUpsertPredicate matchCols matchPredRJoinPredicateBinaryRTableOperationUnaryRTableOperation OrderingSpecAscDesc ROperationROperationEmptyRUnionRInterRDiffRPrjRFilterRInJoin RLeftJoin RRightJoin RSemiJoin RAntiJoin RAggregateRGroupBy RCombinedOpRBinOpROrderBy colPrjListfpredjpredaggListgpred colGrByListrcombOprbinOp colOrdList RPredicate ColumnInfonamedtype ColumnOrder RTupleMData RTableMDatartname rtuplemdata pkColumns uniqueKeys RTimestamp RTimestampValyearmonthdayhours24minutesseconds IgnoreDefaultIgnore NotIgnore RDataTypeRIntRTextRUTCTimeRDateRTimeRDoubleNullrintrtextrutctrdatedtformatrtimerdouble ColumnDType UknownTypeIntegerVarcharDate TimestampDouble RTableName ColumnNameNameRTupleRTableRTabulartoRTable fromRTable rtableToListrtableFromList rtupleToListrtupleFromListisNull isNotNull stdDateFormatgetColumnNamesFromRTabgetColumnNamesFromRTuplegetColumnInfoFromRTabgetColumnInfoFromRTuple getTheTypeheadRTup rtupLookuprtupLookupDefaultgetRTupColValuenvl nvlColValuedecodeColValue nvlRTuple nvlRTable decodeRTable stripRText rdtappendremoveCharAroundRText toRTimestamp toUTCTime fromUTCTimeinstr instrText instrRTextcreateRTimestamptoTextfromTextisTextstdTimestampFormatrTimestampToRTextcreateRTableMDatatoListColumnNametoListColumnInfotoListRDataTypelistOfColInfoRDataTypecreateRDataTyperaggGenericAgg raggStrAggraggSum raggCount raggCountStar raggCountDistraggAvgraggMaxraggMinropUrunUnaryROperationropUresrunUnaryROperationResropBrunBinaryROperationropBresrunBinaryROperationRes isRTabEmpty isRTupEmpty emptyRTable emptyRTuplecreateSingletonRTable createRTuplecreateNullRTuple isNullRTuple rtabFoldr'rdatatypeFoldr' rtabFoldl'rdatatypeFoldl'rtabMap rtupleMaprtupleMapWithKey rtuplesRet getRTuplesRet rtabResult runRTabResultexecRTabResult removeColumn addColumnf runRfilterp runProjectionrunProjectionMissedHitslimitaJ runAntiJoinsJ runSemiJoiniJ runInnerJoinO joinRTupleslJ runLeftJoingetUniqueColumnNamesAfterJoinrJ runRightJoinfoJrunFullOuterJoinurunUnion runUnionAlli runIntersectdrunDiffrAggrunAggregationrO runOrderByrGgroupNoAggList concatRTab groupNoAgg runGroupByrCombrunCombinedROpinsertAppendRTabinsertPrependRTabinsertRTabToRTab upsertRTabrtabsSameStructurertuplesSameStructure deleteRTab updateRTab updateRTuple upsertRTuplegenDefaultColFormatMapgenColFormatMapgenRTupleFormatgenRTupleFormatDefaulteitherPrintfRTable printfRTableeitherPrintRTable printRTable$fOrdRTimestamp$fEqRTimestamp$fNFDataRTimestamp$fFractionalRDataType$fNumRDataType$fOrdRDataType $fEqRDataType$fNFDataRDataType$fExceptionColumnDoesNotExist%$fExceptionUnsupportedTimeStampFormat*$fExceptionEmptyInputStringsInToRTimestamp&$fExceptionConflictingRTableStructures&$fExceptionUniquenessViolationInUpsert$fEqUniquenessViolationInUpsert!$fShowUniquenessViolationInUpsert$fEqConflictingRTableStructures!$fShowConflictingRTableStructures#$fEqEmptyInputStringsInToRTimestamp%$fShowEmptyInputStringsInToRTimestamp$fEqUnsupportedTimeStampFormat $fShowUnsupportedTimeStampFormat$fEqColumnDoesNotExist$fShowColumnDoesNotExist$fEqRTupleFormat$fShowRTupleFormat$fEqFormatSpecifier$fShowFormatSpecifier$fShowOrderingSpec$fEqOrderingSpec$fShowRTableMData$fEqRTableMData$fShowColumnInfo$fEqColumnInfo$fShowRDataType$fReadRDataType$fGenericRDataType$fShowRTimestamp$fReadRTimestamp$fGenericRTimestamp$fEqIgnoreDefault$fShowIgnoreDefault$fShowColumnDType$fEqColumnDType ETLMapping ETLMapEmptyETLMapLDETLMapRD ETLMapBaletlOptabLtabRtabLrdtabRrdtabLbaltabRbal ETLOperationETLrOpETLcOpropcmapColXForm RColMapping ColMapEmptyRMap1x1RMapNx1RMap1xNRMapNxMsrcCol removeSrcColtrgCol transform1x1srcRTupleFilter srcColGrp transformNx1 trgColGrp transform1xN transformNxMYesNoYesNocreateColMappingrunCMetlOpUetlOpBcreateLeafETLMapLDcreateLeafBinETLMapLDconnectETLMapLDetletlResrtabToETLMapping$fEqETLMapping $fEqYesNo $fShowYesNo SetColumnsSet GroupOnPredGroupOnAsColumnAsAggByAggOpSumCount CountDist CountStarMinMaxAvgStrAggGenAgg AggregateAggOn TabExprJoinJoinOn TabLiteralTabLTabExprTabPrevious FromRTableFromByGenBinaryOperation ByBinaryOpByGenUnaryOperation ByUnaryOp ValuesClause TabSourceTabSrc UpdateColumns UpdateCols WhenMatchedWhenMatchedThenMergeMatchConditionMergeOn MergeSourceUsing MergeInto InsertSourceValuesRTuples IntoClauseInto RelationalOpFilterSelectAggGroupByJoinLJoinRJoinFOJoinSemiJoin SemiJoinPAntiJoin AntiJoinP IntersectUnionUnionAllMinusMinusP GenUnaryOp GenBinaryOpOrderByDeleteUpdateInsertUpsertROpExprROpStart:. ByDelPredWhereByPredFilterBy RemoveSrcCol RemoveSrc DontRemoveSrcOnRTableOn ByFunctionByToColumnTargetColMappingExprSourceColMappingEmptyNamedMap NamedResultNamedResultName ETLOpExprEtlCEtlRETLMappingExpr EtlMapStart:->:=>takeNamedResult evalJuliusjuliusToRTable runJuliuseitherRunJuliusjuliusToResultrunJuliusToResulteitherRunJuliusToResultrunETL eitherRunETLaddSurrogateKeyaddSurrogateKeyJ appendRTable appendRTableJCSVColumnToRDataTypeErrorCsvFileDecodingError CSVOptions delimiter hasHeaderColumnRowCSVcsv readCSVFilereadCSVreadCSVwithOptions writeCSVFilewriteCSV printCSVFileprintCSVcopyCSVcsvHeaderFromRtableheadCSVtailCSV selectNrowsprojectByIndex $fRTabularCSV$fExceptionCsvFileDecodingError$$fExceptionCSVColumnToRDataTypeError$fEqCSVColumnToRDataTypeError$fShowCSVColumnToRDataTypeError$fEqCsvFileDecodingError$fShowCsvFileDecodingErrorversion getBinDir getLibDir getDynLibDir getDataDir getLibexecDir getSysconfDirgetDataFileNameString Text.Printfprintfghc-prim GHC.TypesTrue4unordered-containers-0.2.14.0-7xegloYXKrQFs2wz1wk2kBData.HashMap.InternalHashMap time-1.9.3 Data.Time.Clock.Internal.UTCTimeUTCTime GHC.MaybeNothing text-1.2.3.2Data.Text.InternalTextFalse Data.EitherEitherGHC.IOcatchGHC.Exception.Type Exception