write_agilent_amx writes a copy of an 'OpenLab CDS' acquisition method with a new pump gradient, flow, stop time, post time or column temperature. See ?write_agilent_amx.read_metadata now reads the metadata of chromatogram files into a table, skipping the data for most formats that are slow to read. See ?read_metadata..qgd files with MRM scans can now be read. The scans are returned as MS2, with a precursor_mz column. See ?read_shimadzu_qgd..d directories, as MS2 with a precursor_mz column. See ?read_agilent_d.centroid = FALSE, which skips the vendor's peak picking. See ?read_thermoraw..lcd files can now be read: MS1 and MS2 profile spectra, and a TIC per acquisition event. See ?read_shimadzu_lcd..lcd files, such as those from the LCMS-2020, can now be read: centroided and profile MS1 spectra and a TIC per acquisition event. See ?read_shimadzu_lcd.polarity, which write_mzml records in the mzML..lcd files now give each scan's type, such as "MRM", in scan_info$scan_type. 'Shimadzu' .qgd spectra now have a scan_info attribute too.write_mzml now records the collision energy of MS2 spectra where it is known, as for 'MassHunter' files or spectra read by read_mzml. See ?write_mzml.write_mzml now writes MRM scans as one selected reaction monitoring chromatogram per transition. Previously they were MS2 spectra, and scans monitoring several precursors lost them. See ?write_mzml..raw files read with parser = "chromconverter", and the LastModifiedDateTime of 'ACAML' files. Previously, some were hours off or NA.write_mzml now writes startTimeStamp in UTC, and write_andi_ms no longer errors when run_datetime is text.time_range of DAD data from 'Agilent' .uv, .D, .dx and .rslt files is now correct in long format and as a data.table. Previously it held row numbers..ms files read with the chromconverter parser no longer give an error in wide format, which read_chroms uses by default when 'Entab' is not installed..d directories whose MSScan.xsd prefixes its type names with a namespace, which previously gave an error..qgd files now report "MS" as their detector, so write_mzml accepts them without wrapping them in a named list.~ now work for 'Shimadzu' files and in call_rainbow and call_entab, including through read_chroms.read_mzml(parser = "mzR") now returns a list of streams like the RaMS parser, adding MS1, MS2, TIC and BPC to DAD. The DAD chromatogram, now $DAD, has retention times in minutes rather than seconds, and takes its wavelengths from the spectra rather than the scan window, so DAD files without one, such as those write_mzml writes, can now be read.chroms from mzML and 'Thermo' RAW files is now a list of 2D chromatograms, one per stored trace, with precursor_mz and product_mz attributes. Use data_format = "long" for the previous table..d directories by default, and from 'OpenLab' .dx files with what = c("MS1", "TIC", "BPC"). See ?read_agilent_d.call_rainbow gains centroid and hrms arguments to read the centroid and profile mass spectra of 'MassHunter' .d directories, which 'rainbow' otherwise skips. See ?call_rainbow.read_mzml now returns extracted ion chromatograms with what = "EIC" or "EIC_MS2", for the masses given by mz and ppm. See ?read_mzml..dx files and .rslt/.sirslt directories are now in mAU. To convert DAD data read by earlier versions, multiply it by its intensity_multiplier attribute..lcd files are no longer about one sampling interval early (thanks to @JustCoderWorld, #55).read_chroms now recognizes 'Agilent' .d, 'OpenLab' .rslt/.sirslt and 'Waters' .raw directories without format_in..dx files whose injection volume unit is written with the Greek letter mu, rather than the micro sign, now report their injection volume..lcd and .gcd files and Allotrope (ASM) files now report the acquisition software, which write_mzml records in the mzML. ASM files also report software_version..qgd files now report the software release (e.g. 4.45) as software_version. The file-format version they reported before is now file_version.instrument, polarity, and the ion source, analyzer and detector (as ms_params) recorded in the file.data_format attribute of "wide" rather than "long".call_rainbow(by = "name") no longer errors when 'rainbow' finds a trace with no detector type, and names files in their original case rather than upper case. Asking what for an absent detector now gives a clear warning.read_peaklist now reads peak tables from 'Agilent OpenLab CDS' .rx files with format_in = "openlab", named by signal from the .acaml file beside them.get_element to extract one element, such as "dad" or a pump pressure trace, from each sample of a chrom_list read with several what values.read_peaklist now warns when two samples get the same name, as read_chroms does.c() now returns a chrom_list that is also a list when combining chrom_list objects, such as the output of two read_chroms calls. Previously the result was not a list, so chromatographR's preprocess rejected it.tools::R_user_dir("chromConverter", "config") and survive package updates. You may be asked for them once more after upgrading.path_out contains spaces.read_chroms and read_peaklist now return files found in a directory in acquisition order, if every file records its run time, rather than alphabetically. sort_by = "none" restores the previous order.rt, start, end, area, height and FWHM (NA where the file records none), in both read_chroms and read_peaklist. 'ChemStation' width is now FWHM. Use peaktable_format = "original" to return all the columns read by the parser.read_peaklist tables from every format now have a numeric lambda column after sample, giving the signal's wavelength (NA where none is recorded)."254") wherever the file records one, both in read_peaklist and in 'ChemStation' .D directories. Tables without a wavelength keep their names.?read_cdf.time_unit is "Minutes". Retention volumes (mL) are unchanged.U2, U5 and U6 as Start, End and SubstanceIndex..dx files now apply the recorded time-zone offset, so they are true UTC rather than local time mislabeled as UTC.detector_range now holds only a numeric wavelength range, as for 'ChemStation' since 0.10.0. Channel descriptions and single wavelengths that 'Waters' .arw, ANDI, 'Lumex' MDF and ASM files put there are now signal_descriptor or wavelength.time_range, are now numbers, both on chromatograms read from text formats and in extract_metadata output. So are the wavelengths of 'Shimadzu' .lcd traces, previously text such as "260nm", including the lambda column of long-format chromatograms. An injection volume written with its unit, e.g. 17 µl, is now split into the number and sample_injection_volume_unit.extract_metadata now returns one row per sample, retaining the values its chromatograms agree on. The by = "chromatogram" argument restores one row per chromatogram (in nested lists).what = "chroms" from read_varian_sms, which returned an internal table rather than a chromatogram. Use what = "TIC" or "BPC".extdata/ladder.txt is now extdata/alkane_ladder.txt, documented at ?alkane_ladder. Update the path in system.file calls.read_agilent_amx returns the DAD's peak width setting as peakwidth_code, replacing peakwidth_nm. The value is a stored code, not a width in nm.read_agilent_amx now names solvent channels A–D rather than Channel_A, matching the gradient's pct_A columns, and DAD signals A, B, etc. rather than SigA.vignette("chromConverter"), which works through a calibration series of benzoxazinoid standards now included as example data. See ?benzoxazinoid_standards.read_chemstation_method to read the gradient, solvents and other instrument settings from 'Agilent ChemStation' methods, including the copy a .D directory may hold, whether stored as register files or as 'RapidControl' XML.read_shimadzu_method to read the oven program, gradient and other instrument settings stored in 'Shimadzu' GC (.gcd), LC (.lcd) and GC-MS (.qgd) files.read_chemstation_logs to read the instrument errors, aborted runs and pump pressures recorded in 'Agilent ChemStation' sequence logs, including every log in a folder..D directories now return pump pressure, flow, solvent composition and temperature traces from LCDIAG.REG with what = "instrument". See ?read_agilent_d.read_agilent_d and read_peaklist now read 'ChemStation' Result.xml files where present, which unlike Report.TXT record where each peak starts and ends.read_asm now reads mass spectra from ASM GC-MS files, and peak lists and instrument traces (e.g. pump pressure, flow rate, temperature) from any ASM chromatography file, chosen with a new what argument. read_peaklist accepts format_in = "asm".read_peaklist now reads peak tables from 'ANDI chrom' files with format_in = "cdf".read_thermoraw now accepts format_out = "data.table", like every other parser.rainbow parser now report most of the metadata the parser supplies, including instrument, operator, detector_model, wavelength and instrument_modules..dx files read with the rainbow parser are now returned as POSIXct in UTC, applying the time-zone offset the file records..dx files now report their injection volume..ch, .uv and .it traces now report the channel they were recorded on as channel_id (e.g. DAD1A or PMP1A), and its module as detector (e.g. DAD)..ch traces and 'ChemStation' peak tables now report wavelength and bandwidth from their signal descriptor (e.g. Sig=254,8); peak tables also report detector, channel_id and the descriptor itself as signal_descriptor.chrom_listssubset() now selects the chromatograms of a chrom_list by their metadata, e.g. subset(chroms, sample_name == "S1"). See ?subset.chrom_list.add_metadata to attach sample metadata from a table to a chrom_list, matched by name, for use with subset() and extract_metadata.extract_metadata now accepts an element of a nested field in what, such as "SampleLabel" or "acaml_metadata.SampleLabel", and returns that column alone.print() of a chrom_list now shows numeric metadata to getOption("digits") significant digits, set with a new digits argument that summary() and extract_metadata() also take.write_mzml now writes MS2 spectra, interleaved with MS1 in scan order, each with its precursor m/z and a reference to the MS1 scan before it. MS2 was previously skipped.centroided argument lets profile data be marked as profile. All spectra were previously assumed to be centroided.instrument_info argument still overrides the instrument.write_andi_chrom.read_cdf(peaktable_format = "original") now keeps the text columns of 'ANDI chrom' peak tables, such as peak_name, which were previously dropped.write_andi_chrom now record the correct time unit, run length and detector range, and the exported signal's wavelength, which read_cdf reads back.NA and no TIC is returned.detector_y_unit, which was always empty, and files from other software also report their mass unit in detector_x_unit.write_mzml to keep both.write_andi_ms now follow the ANDI MS specification more closely, recording each scan's actual mass and time range, the intensity unit, polarity, instrument, software and acquisition mode. See ?write_andi_ms.ms_params in write_andi_ms no longer drops the defaults of the others.call_rainbow now works when format_in is not supplied, which failed on R 4.2 and later.rainbow parser no longer lose their retention times when it returns fewer times than rows, as for compressed 181 .ch files. The time axis is rebuilt, with a warning.Andi-MS format).id (sNA) or characters an ID cannot hold, and sample_name now also names the <sample> element. Metadata values are now XML-escaped.run_datetime no longer fail schema validation.write_mzml now errors on a table without mz (or lambda) and intensity columns, instead of writing a corrupt file..lcd chromatograms (scale = FALSE) now report detector_y_unit as NA where the channel's calibration factor is not 1, since those values are not in a defined unit..lcd files now carry the scaled attribute..qgd scans whose intensities are more than 4 bytes wide can now be read, with a warning that this decoding is unverified..lcd peak tables stored as Peak Table-100 and similar now read correctly; every peak after the first was misread..lcd peak tables no longer include mass spectrometry tables (Mass Peak Table, Compound Peak Table), which were misread.read_peaklist no longer returns the peak table of a PDA channel twice for 'Shimadzu' .lcd files..lcd and .gcd peak tables now carry the file's metadata, such as sample_name, instrument and run_datetime, and their channel's wavelength (and bandwidth, for channels extracted from PDA data), as the chromatograms do..gcd, .lcd and .qgd files now report their method's settings as method_params, and the mass spectrometer settings of GC-MS and quadrupole LC-MS runs, such as ion source temperatures, as ms_params. See ?read_shimadzu_method..ch files written by 'Asterix ChemStation' are now decoded as 8-byte doubles. They were incorrectly read as 4-byte floats, which compressed the signal into tiny values near 0.001.read_agilent_amx now warns when a method lacks a requested module, such as a DAD, and returns the others; asking for a missing module alone gives a clear error.solvents table from read_agilent_amx now returns the class that format_out asks for.read_agilent_amx signal tables now include each signal's reference wavelength and bandwidth, as reference_nm and reference_bandwidth_nm.slitwidth_nm from read_agilent_amx is now the slit width in nm (e.g. 4). It previously held the stored code (2).read_agilent_dx(what = "instrument") now gives a clear error for an archive without instrument data.run_datetime, and read sample_name, method and the injection volume correctly.extract_metadata no longer splits the source_file of 'ChemStation' peak lists into one column per sample.sample_name and run_datetime, as the chromatograms do..ch and .uv traces now report the volume actually injected, which can differ from the volume the sequence requested..D directory now get distinct names, e.g. RUTIN_2.dad1A, instead of all sharing the directory or sample name." or :, with _. Sample names containing them previously failed to export on Windows.read_chroms adds to its list of samples."UV 1_280") where their detection types don't tell them apart, in the list and in the detector column.detector_reference, detector_y_unit and time_unit now describe each chromatogram, rather than being copied from the file's other traces. LC-MS files no longer warn that row names were discarded.detector_model is now reported, as are the method and injection volume of GC files and the sample name and run time of files from the 2022 releases.wavelength setting.'names' attribute error.metadata_format = "raw") now keep the file's own field names, e.g. injection time and analyst, and describe each chromatogram rather than the whole file.read_chromeleon now honors decimal_mark = "," when it is given explicitly. Previously, the commas were stripped, inflating every value.read_peaklist now warns about files it cannot parse and drops them, rather than returning the error message as a peak table.read_mzml now returns DAD data in the correct class as requested by format_out..lcd files are now scaled by the calibration factor as well as the value factor, so the intensities match those reported by 'LabSolutions'. The calibration factor converts the encoded integers into the base unit of the detector, and is stored alongside the raw data in the Chromatogram Status stream. The parser was instead taking it from the copy of the 2D Data Item under LSS Data Processing, where it is always 1. Channels where it is not 1 were off by a constant factor, such as ~42x for an SPD-20A UV detector and ~310x for an RID-10A refractive index detector, so any factor applied by hand to match 'LabSolutions' should now be removed. Older files, written by 'LCsolution' rather than 'LabSolutions', have no 2D Data Item at all, so neither factor reached them; both are now read from the status record, changing the scale of those chromatograms by up to ~5000x. scale = FALSE still returns the unscaled integers.apply(x, 2, as.numeric). Because retention times entered that table as character (from the rownames), the coercion routed every column through a character matrix, formatting each intensity with getOption("digits") and so rounding it to 7 significant figures. Long-format intensities now match the wide-format values exactly. Exported files were affected too, since write_mzml and write_andi_ms reshape to long format before encoding.format_out = "matrix", which is the default, read_shimadzu_qgd, read_chemstation_ms and read_cdf returned MS1 as a matrix, and read_varian_sms returned a data.frame but recorded format_out as matrix. Long spectral data has no useful matrix representation: the coercion promoted scan and rt to double alongside the intensities and gave up $. All four now return a data.table for this value of format_out, and record it. data.frame and data.table are returned as requested, and the two-dimensional streams (TIC, BPC) still honor matrix..lcd or .gcd file now reports its detector as DAD rather than PDA, matching the vocabulary used by every other parser. read_shimadzu_lcd also returns the stream under the name DAD rather than pda, so a result indexed as x$pda needs updating --- though only where more than one stream was requested, or collapse = FALSE, since a single-stream read returns the object itself, unnamed. what takes either spelling, so the two are interchangeable throughout.parser is not specified, reading a 'Waters' .raw directory now requires the rainbow Python module, and reports how to install it when it is missing. Auto-detection previously fell back on the internal parser, which reads only the analog _CHRO traces and reported nothing about the MS and PDA data it had skipped, so a file with three detectors could come back with one. The fallback also depended on a test that started Python for every file, including the formats that need no Python at all. rainbow-api is requested automatically, so only hand-managed environments are affected; parser = "chromconverter" still selects the internal parser for files that hold analog traces alone.read_agilent_rslt function to read whole sequence of files from OpenLab and automatically attach corresponding metadata from the acaml file.read_shimadzu_lcd can now read mass spectra from .lcd files, with what = "MS1", "MS2", or "MS" (to retrieve both levels). Two types of containers are supported: QTFL RawData (centroided quadrupole time-of-flight) and TLM Raw Data (triple quadrupole full scan, product-ion scan, MRM and SIM). what = "TIC" reads the total ion current from either container. QTOF m/z are computed from the TOF calibration stored in the file, with the mass correction 'LabSolutions' cached for the run folded in, which reproduces its mass axis exactly. Where a file carries no cached correction, the calibration is refit against the lock mass reference ions found in the data instead, which is good to a few tenths of a ppm. A new sparse argument controls whether zeros are dropped from triple quadrupole profile spectra.sort_by argument to read_chroms to control chromatogram order. Options are "none" (default), "acquisition_time" (using run_datetime from metadata), and "file_time" (using file modification time). The default will change to "acquisition_time" in a future release.bin_width argument to call_rainbow as an alternative to precision, for m/z grids that are not a power of ten (e.g. bin_width = 0.5). precision is unchanged and remains the default.summary method for chrom_list objects, which returns what print displays as a table: one row per chromatogram, with the sample it belongs to, its dimensions and relevant metadata fields, such as wavelength or detector_range for an optical detector, scan_type, polarity, precursor_mz, product_mz and mz_range for a mass spectrometer.expand argument to extract_metadata for the nested metadata fields, whose value is itself a list or table rather than a single value per chromatogram: the ms_params instrument settings, the acaml_metadata injection record read_agilent_rslt reads from the .acaml file, or the whole vendor list when metadata_format = "raw".collapse argument to extract_metadata, which renders a field holding more than one value as a single comma-separated string instead of spreading it over numbered columns (time_range1, time_range2).detector argument to extract_metadata to select which detectors to include (e.g. detector = "UV" or detector = c("UV", "MS")), matched case-insensitively against each chromatogram's detector attribute. This is useful for lists containing more than one detector per sample, such as those returned by the rainbow parser.write_andi_ms is now exported, like write_andi_chrom and write_mzml. It was previously reachable only through write_chroms(what = "MS1") or read_chroms(export_format = "cdf"), which write a whole list of chromatograms and offer no control over the file name or the instrument settings.write_chroms(export_format = "cdf") now forwards ... to the underlying writer, as the mzml exporter already did. This makes the ms_params argument of write_andi_ms and the lambda argument of write_andi_chrom usable when writing a batch of files.rainbow, olefile or Aston) is actually called, so the formats read by the internal parsers no longer require an internet connection. When Python is needed and the package index can't be reached, chromConverter now falls back on a previously cached environment instead of failing.rainbow or olefile parsers no longer installs the 'Aston' requirements or constrains which version of scipy you can have.configure_python_environment so it accepts the parser argument it is called with, and removed its interactive prompts, which failed in non-interactive sessions..uv file went from ~9 s to ~0.57 s..qgd file went from ~56 s to ~1 s, and reading a PDA stream from an .lcd file went from ~7 s to ~1 s.read_varian_sms for increased speed (~8x) through vectorization. Reading STRD15.SMS (2.4 MB, 935k MS1 rows) drops from ~13 s to ~1.7 s. The stream is also bounded by the end of the MSData section rather than the end of the file, which reduced peak memory requirements for files carrying a large tail of peak tables and results.write_mzml for increased speed (~1.4x) and lower memory use. The spectra are now sliced out of the long-format table in place instead of being copied into a list of per-scan tables, and the byte offsets for the index are accumulated as the file is written rather than probed with seek() once per scan. Writing 3432 scans (935k points) drops from ~1.24 s to ~0.87 s, with peak memory falling from ~384 MB to ~339 MB.data_format = "long" (or format_out = "data.table", which implies long format), as well as the mzML and ANDI MS writers, which reshape to long format internally.source_sha1 metadata attribute, is now computed once per file instead of once per chromatogram. Formats that return several chromatograms from one file hashed it again for each of them, which dominated the read for large files: a 40 MB .qgd drops from ~2.2 s to ~1.3 s.extract_metadata reports cannot drift apart. Five fields had drifted and are renamed: software_name is now software ('Shimadzu', 'Varian' SMS and read_agilent_rslt); run_date is now run_datetime and injection_volume is now sample_injection_volume ('Thermo' RAW); time_start and time_end (or end_time) are now the two ends of time_range (ANDI, 'Lumex' MDF and 'Varian' SMS); and the two formats that record a scan count, 'Varian' SMS and ANDI MS, now both report it as n_scans, matching the n_ prefix used for counts throughout the package, rather than no_scans in one and ms_params$n_scans in the other. extract_metadata accepts the old names and maps them to the new ones.parser attribute is now always spelled chromconverter. With metadata_format = "raw", some readers reported chromConverter instead.sample_amount is no longer copied from the injection volume ('Shimadzu' ASCII, 'ChemStation' .ch, .uv and .ms files, 'ChemStation' report files, and 'MassHunter'). None of these records a sample amount, so it is now NA. 'Lumex' MDF likewise no longer reports an injection volume and amount of 1, which the file does not record.detector_id is renamed as detector_model. Every format that fills the field supplies a module or a model number, and says as much internally: detector_model for 'ChemStation', 'OpenLab' and 'Chromatotec' (G1315B, HP G1530A), detector_model_number for ASM, detector_name for ANDI (9065 UV-DAD) and Detector Name for the 'Shimadzu' ascii exports. None supplies a serial number or any other identifier of a particular unit, so the old name was misleading. extract_metadata accepts detector_id and maps it to the new name.detector_range is now reserved for the numeric wavelength range recorded by .uv files. For 'ChemStation' versions 30 and 130 the signal descriptor was previously reported in this field, and is now reported as signal_descriptor.detector field is now NA for 'ChemStation' .ch files. These files do not record a detector type; the field previously reported the detector module, duplicating detector_id.read_acaml now also returns the injection volume (InjectionVolume, InjectionVolume_unit) and the acquisition software name and version (Software, SoftwareVersion).wavelength attribute of a 'Shimadzu' 2D chromatogram is now NA rather than an empty string when the channel records none, as a refractive index or FID trace does. An empty string printed as a blank cell instead of as a missing value..lcd and .gcd files now report the instrument the file was acquired on, rather than the detector module of whichever trace you are looking at: one run on one HPLC previously came back as SPD-20A on two channels and RID-10A on a third. instrument is taken from the SystemInformation stream (Instrument2, HPLC RID, GC-2014), the same string the ascii exports report as Instrument Name, so a run exported both ways now agrees. The module is now reported as detector_model. A mass spectrometry trace has no module of its own, so it reports whatever unit SystemInformation lists for the mass spectrometer instead. That is a model number on newer software (LCMS-9030), but older versions list the generic platform name (e.g., LCMS-3030 for every triple quadrupole)..lcd or .gcd trace was read from (LC.1.1.DET.1.CH#1, PDA.1.1.PDA.1.3D) is now reported as channel_id, (replacing detector_id). This field is used to name the peak table belonging to a trace (PT-LC.1.1.DET.1.CH#1), so the two can still be matched up..lcd, .gcd and .qgd files now report file_version, the version of the container format (5.01 for files written by 'LabSolutions'; absent in the older files, which report only a software_version of 1.x).segment_start_time and segment_end_time within ms_params. time_range gives the span of the whole run, as it does for every other format.aston parser is deprecated and will be removed in a future release. 'Aston' has been unmaintained since 2020. It is now used only by sp_converter to read 'Agilent MassHunter' .sp files (format_in = "masshunter_dad"), and read_chroms selects it automatically only as a last resort, when no other parser can read the file. Please use the internal chromConverter parsers or the entab parser (by the same author as 'Aston') instead.uv_converter is now defunct; use read_chemstation_uv or the entab parser instead. The aston binding for format_in = "other" has also been removed; this format is still handled by the entab parser. Both relied on an 'Aston' reader that requires scipy < 1.14, which would otherwise constrain the Python environment for every user.dat argument in read_chroms. Instead, chrom_lists can be combined with c().data_format argument of read_peaklist and read_chemstation_reports to peaktable_format. This argument selects chromatographr or original peak table layout, so it had nothing to do with the data_format argument of the chromatogram readers, which selects wide or long format. peaktable_format is the name already used for this option by read_shimadzu. The old name still works but warns, and will be removed in a future release.read_sz_lcd_2d and read_sz_lcd_3d are no longer exported. Each reads a single stream of an .lcd file and neither has to be called directly: read_shimadzu_lcd(what = ...) picks the reader a stream needs. Their documentation remains, since each records the layout of the stream it reads, but is marked internal and no longer appears in the reference index.format_out = "data.frame", which returned a data.table for any parser that assembles its result as one. The conversion tested the object with inherits, and a data.table inherits from data.frame, so the conversion was skipped.rainbow parser, which raised read() no longer takes precision on every call once rainbow-api v1.5.0 was released. v1.5.0 split precision into bin_width (the m/z grid, in daltons) and display_precision (label rounding, in decimals); chromConverter now derives both from precision, so the argument and the data it returns are unchanged. v1.5.0 is now the minimum required version.nchar() and toupper() failed on them and grepl() could not match them. Accented characters in a path or sample name are now preserved rather than mangled.input string 1 is invalid UTF-8) when printing a chrom_list read from a 'Shimadzu' .lcd written in a non-Latin locale. The hex-encoded @StoX@ fields, which hold the method and batch paths, skipped the Latin-1 decoding applied to the rest of the file's strings. Undecodable bytes are now replaced with ?, and print.chrom_list repairs whatever it is handed, so one mangled path cannot take down the summary.thermoraw, openchrom, agilent_dx and agilent_amx parsers deleted the whole session temporary directory on exit, instead of just the files they created. This behavior could potentially create conflicts with other packages. Each call now gets its own directory inside the session temp directory which is cleaned up on exit.reticulate::py_install("rainbow"), which installs an unrelated package; the 'rainbow' module comes from rainbow-api.detector_id for 'ChemStation' version 130 files.sample_position metadata field for 'ChemStation' 179 files (.ch and .it).POSIXct instead of being attached as an unparsed string, which extract_metadata reported as NA.read_agilent_dx, read_agilent_amx and read_agilent_rslt to fail when path_out was supplied.read_chemstation_reports, which listed the accepted values as chromatographr or chemstation. The second value has always been original, so following the documentation raised an error.NA for every export not written by a machine using a 12-hour month-first date format. Note that the times in an ASCII export are local to that machine, which does not record its time zone, whereas .lcd files record the acquisition instant in UTC.detector and no detector_range; it now reports DAD and the wavelength range the detector covered.read_shimadzu_lcd for .lcd files that do not contain a 2D Data Item, which failed with 'names' attribute [4] must be the same length as the vector [2]. This bug seems to affect older files, which store their chromatograms under LC Raw Data rather than LSS Raw Data.read_shimadzu_lcd so it can return PDA data in long format. read_shimadzu_lcd(what = "PDA", data_format = "long") previously failed with an error about a missing lambda column, because the reshaping step was called with the wrong target format.format_out now reaches the peak tables of 'Shimadzu' .lcd and .gcd files. read_shimadzu_lcd(what = "peak_table") and read_shimadzu_gcd(what = "peak_table") accepted the argument and then dropped it, so a table always came back as a data.frame. data.table is now returned when asked for --- matrix, which has no useful representation for a peak table, resolves to data.table as it does for mass spectra.~, which is not expanded by Python..lcd data streams, which was read as a signed 2-byte field rather than the 4-byte field it is. A 2D chromatogram with 32,768 to 65,535 points failed with invalid 'length' argument, and one with more than 65,535 points was truncated to the remainder. Files below that threshold, such as the 30,000-point chromatograms in the test suite, were read correctly.run_datetime for 'Varian SMS' files is now the acquisition start time, as a single value rather than a start/end pair.read_varian_sms for format_out = "data.table", which failed previously with an error. The TIC and BPC returned by this parser also had their intensity column named tic/bpc instead of intensity for this value of format_out.instrument and method metadata from Varian SMS files (read from the InjectionLog section).read_cdf for 'ANDI MS' files in which every scan holds the same number of points (common when the instrument scans a fixed mass range). The retention times arrived as a matrix and were split into one column per scan, so a 20-scan file returned a table with rt.1, rt.2, ... rt.20 columns instead of a single rt column. With ms_format = "list" the same files returned a list of individual numbers rather than a list of spectra. Files with a varying number of points per scan were unaffected, and their output is unchanged.read_cdf no longer opens the netCDF file twice, and the peak table returned for 'ANDI chrom' files is no longer transposed when it holds a single peak.read_cdf(what = "peak_table") on an 'ANDI chrom' file that holds no peak table now warns and returns the other streams that were asked for, rather than failing with value for 'peak_table' not found.write_mzml now warns that MS2 spectra are skipped rather than counting them in the spectrumList header and writing none of them. Writing MS2 is not supported yet.write_chroms(export_format = "mzml"), which failed for every file unless what was given explicitly. The streams to write are now inferred from the data, as they are when write_mzml is called directly.write_mzml no longer writes a one-dimensional chromatogram as DAD spectra, and points to write_andi_chrom instead. mzML stores scans of (m/z or wavelength, intensity), so a single trace has no axis to put in one: it was written as a single-point spectrum per retention time, which for a 66,000-point trace meant 66,000 scans, as many warnings about empty ranges, and a 128 MB file. The trace is skipped with a warning if other streams were requested, and reported as an error if it was the only one, since skipping it would leave an empty file. TIC and BPC are unaffected, since the format has terms for those MS-derived summaries and they are written to the chromatogram list rather than as spectra.write_mzml now throws a more informative error when handed a single chromatogram whose detector attribute is missing, NA, or names a detector it has no mzML stream for, rather than failing with EXPR must be a length 1 vector or quietly writing an unnamed stream.fileChecksum written into indexed mzML files, which was the SHA-1 of the first line of the file (<?xml version="1.0" encoding="UTF-8"?>) rather than of the file itself, because the digest was taken over a multi-element character vector. Files are now checksummed as required by the mzML specification, over the bytes up to and including the opening <fileChecksum> tag, and the file no longer has to be read back into memory to do it.indexList of mzML files. <indexListOffset> pointed one byte before <indexList>, and every offset in the DAD spectrum index pointed at the newline preceding its <spectrum> element rather than at the element. Offsets are now counted as the file is written instead of being probed with seek(), which is unreliable on a connection opened in text mode and ignores the write buffer.<offset> pointed four bytes before its <chromatogram> element, and its idRef named the element's index rather than its id, so no entry in the index resolved to the chromatogram it was meant to locate.MS:1000580 ("MSn spectrum") even though ms level was 1.write_mzml(compress = FALSE), which was ignored for spectra (though not for chromatograms), since the argument was never passed on.count attribute of <spectrumList>, which was always written as 1 for data read as a data.table.sample_name, source_file or source_sha1 attributes: a missing field collapsed the sprintf that builds the header, dropping the <mzML> element itself. The header read its metadata from MS1, or from DAD when MS1 was not requested, without checking the stream was still there --- a DAD trace skipped earlier leaves nothing to read from. It now reads from a stream that is actually being written, and a missing sample_name is an error naming the argument that supplies one.count attribute of <spectrumList> when the TIC starts before the first MS1 scan. Those leading retention times are written as empty spectra, so the file holds one spectrum per TIC point, but the count was taken from the MS1 table and fell short by the number of padded scans.index attribute of the DAD spectra, which with indexed = FALSE restarted at zero and repeated the numbers the MS1 spectra already used. The starting value was parsed from the id of the last spectrum-index entry, and those entries carry an id only when the file is indexed; it is now the count of spectra already written.LF on all platforms.switch, and attributes like source_file, source_sha1 and parser are no longer repeated in each one. The change is internal, but it resolves several problems listed below.metadata_format, which several readers mishandled. metadata_format = "raw" errored for 'Chromeleon' files and returned NULL instead of a chromatogram for the rainbow parser; the entab parser and read_shimadzu ignored the argument altogether; and parser = "entab" with format_in = "other" returned NULL. The argument is now resolved in one place instead of separately by each reader.read_peaklist returns, so extract_metadata can see it.detector_range for 'Chromeleon' 3D files, which reported only the lower end of the scan range. For 2D files, which record no scan range, it was a zero-length value that disappeared from extract_metadata instead of reading NA.sample_info.xml.read_mzml recovers are now attached as attributes, so extract_metadata and print.chrom_list can see them, and the metadata element carrying them is no longer counted as a chromatogram. The run_datetime, time_range, time_unit and detector_range fields previously came back as NA for mzML files even though 'RaMS' had parsed them, which also meant read_chroms(sort_by = "acquisition_time") could not order them. The metadata element is still returned in full, since it carries several fields with no attribute equivalent.extract_metadata now reports them as empty instead of dropping them altogether. scaled was accepted by every reader that takes a scale argument and then dropped by thirteen of the twenty field maps, so it was missing for most formats; metadata_format = "raw" recorded source_file and source_sha1 but not format_out or source_file_format; and 'Thermo' RAW files lost any field that the mzML conversion recorded but the converter's own metadata output did not.source_file_format attribute is now set correctly for every format. It was previously missing for some formats, and named the parser or an intermediate file for others.extract_metadata now reports the injection volume for every format that records one, and the acquisition software for 'Shimadzu' and 'Varian' files. These were previously ignored, because the formats did not agree on what to call them.extract_metadata now returns a row for every chromatogram, however deeply nested, and reads sample-level attributes from the list enclosing a sample's traces as well as from the traces themselves. Previously only the top level of the list was examined, so nested traces, and any metadata held on the list grouping them, were left out of the table. A field that varies from trace to trace, such as detector in a multichannel file, stays with the trace; where the traces agree, the value on the enclosing list is used, since it describes the sample as a whole.extract_metadata no longer reports a multi-valued field as missing. The check tested the requested names against the columns of the assembled table, where a field holding several values appears as product_mz1, product_mz2, ... and so never under the name that was asked for.extract_metadata now matches attribute names exactly. Previously a requested element could be filled in from a different attribute that merely started with the same characters, so a chromatogram with no detector attribute could report its detector_y_unit as its detector.extract_metadata now returns NA instead of a metadata frame with only a name column when none of the requested metadata elements are found.print.chrom_list now handles lists holding more than one trace per sample, such as a multichannel 'Shimadzu' file or an 'Agilent' .dx read with what = c("chroms", "dad"). Traces are grouped under the sample they belong to, however deeply nested, and attributes shared by all of a sample's traces are shown in that sample's block header instead of being repeated on every row. Previously only the top-level elements were counted, so the chromatogram count was wrong and only the first trace of each sample was shown.print.chrom_list formatting: datetimes print as timestamps rather than raw epoch seconds; the header wraps to the width of the console, breaking between fields; long values, such as a 'Windows' method path, are shortened from the middle; and a field that is empty for every chromatogram is dropped. print no longer errors when none of the requested cols are present or when n is negative, and n now defaults to 10, as documented.read_chromsread_chroms. A single internal table now records which formats it can read and, for each one, the file extension, and the reader each parser uses. These facts were previously spread across the argument list, an if/else chain, and several utility functions (check_parser and format_to_extension). The change is internal, but it resolves several problems listed below.... are now matched against the arguments the selected parser actually accepts. An unrecognized argument previously made every file fail, surfacing from inside try() as a warning with an unreadable message; such arguments are now ignored with a warning naming them.format_in is not supplied and the type of a file cannot be recognized, read_chroms now says so and asks for a format, rather than failing with argument is of length zero.read_chroms now gives an informative error when no parser is available for a format, instead of failing with missing value where TRUE/FALSE needed.read_chroms now names it instead of reporting its position in the list.suppressWarnings and was invisible to callers handling the warning.format_in accepts an alias as readily as the format's own name, and the two behave identically: rslt and sirslt for agilent_rslt, openlab_dx for agilent_dx, chemstation_fid for chemstation_ch, andi for cdf, and allotrope for asm.shimadzu_ascii, csv, asm, openlab_131 and chemstation formats are now matched to the correct file extension, instead of falling through to a pattern matching any file containing a ..metadata_format argument now reaches the agilent_dx, agilent_rslt, shimadzu_lcd, shimadzu_qgd, cdf and entab parsers. Previously, metadata_format = "raw" had no effect for these formats.sample_names = "sample_name" named every sample with the literal string "NULL" and then warned about duplicate names. This was because the sample_name attribute is attached to the individual chromatograms rather than to the list grouping them, so the lookup came up empty and the resulting NULL was coerced to a string. Samples with no recorded sample name now fall back to the file name, with a warning naming them. A name that is recorded but empty counts as no name, since a parser that finds the field but reads nothing out of it leaves an empty string behind. The traces making up a sample are now also checked against each other: if they disagree about the sample name there is no basis for preferring one over another, so the file name is used instead, again with a warning.[.chrom_list method so that subsetting a chrom_list preserves its class instead of dropping it to a plain list.c.chrom_list method so that combining chrom_list objects with c() preserves the class instead of dropping it to a plain list.read_shimadzu_lcd: (bytes are now read explicitly as ISO-8859-1 rather than relying on system locale via readLines()).prec argument to precision; chromConverter now requires rainbow >= 1.3.0.varian_sms so the example file is downloaded in binary mode (mode = "wb"), preventing file corruption on Windows.sample_position field to extract_metadata.sample_id and vial metadata fields into new sample_position field.sparse argument for rainbow parser (now enabled by default) to remove zeros form long-format MS data.what arguments in read_agilent_d to prioritize DAD data (instead of 2D chromatograms).chrom_list class and print.chrom_list method. Instead of dumping the full contents of every chromatogram, prints a compact metadata summary with configurable columns (cols) and row limit (n)..amx)..Chrom files through the read_chromatotec function.csv files.agilent_dx files so that the original source file is stored in metadata instead of a temp file.read_chroms for duplicated names as they may silently interfere with downstream analyses.write_chroms. This format seems to be the simplest way to get DAD data into OpenChrom.csv files and fixed a bug causing column names to be prepended with "X".write_chroms so it invisibly returns file names of the exported chromatograms.write_chroms..cdf (thanks to @pbulsink for PR #37)..cdf files.write_mzml to correctly handle DAD spectra.write_mzml..ch files to wide format when specified.read_agilent_d when subsetting data with the what argument.rainbow parser are in minutes.data.table object when specified by user.read_agilent_dx and read_themoraw) are actually deleted on completion.extract_metadata function for simplicity..dx files: extraction of DAD and auxiliary instrumental data (stored in .IT files).read_shimadzu_qgd for a 1.4x speedup in the parsing of Shimadzu .qgd files, cutting execution time by 30%.read_shimadzu_lcd for a 2.4x speedup in the parsing of Shimadzu .lcd files, cutting execution time by 60%.write_mzml for massive speed-up when writing mzML files, especially for large MS data.read_agilent_d that were causing possible bug.read_chemstation_uv causing error for long format data.read_agilent_d.data_format attribute.data_format and read_metadata arguments for read_chemstation_csv.data_format attributes for MS data to reflect that they are always returned in long format.data.table output is selected since data.tables do not have rownames.write_mzml causing retention time shifts for BPC and TIC.configure_python_environment function to facilitate configuration of a chromConverter virtual environment or conda environment, though a dedicated environment is no longer required (as of chromConverter v0.7.4).collapse argument causing functions to return vector when format_out is data.frame.format_out is data.table.data.table format in read_shimadzu_ascii..dx file by read_chroms.read_shimadzu ('Shimadzu' ASCII files).sample_names argument in read_chroms: This argument can no longer be supplied with a vector of names. Instead, sample_names can be provided with one of two arguments: basename or sample_name. The default setting (basename) will use the basename of the file, while sample_name will instead use the sample name encoded in the file's metadata.extract_metadata when sub-setting metadata elements.extract_metadata.data.table format option to extract_metadata.call_openchrom.call_openchrom manual concerning OpenChrom installation.read_shimadzu_lcd to infer retention times in Shimadzu 3D Data from Max Plot stream since it is always (?) present.read_shimadzu_lcd to skip parsing of metadata from 3D Data Item when it is not present.read_shimadzu_lcd to include Max Plot stream when parsing 2D chromatograms.read_chromeleon related to inference of decimal separators.decimal_mark argument to read_chromeleon to manually set decimal separator..lcd files..lcd files lacking Data Item streams..lcd files..raw and Agilent .D)Waters chromatograms with lowercase filenames..raw header files..sms) format through read_varian_sms function.read_shimadzu_qgd function..D directories through read_agilent_d function.read_agilent_ms.read_chroms so that format_in comes second after path.export argument from read_chroms. To export files, you now only need to provide an argument to export_format.read_shimadzu_lcd. The function now returns a list of named chromatograms if data_format == "wide" and returns multiple chromatograms in a single data.frame if data_format == "long".scale argument to read_chemstation_uv and read_shimadzu_ascii to toggle scaling of chromatograms.file arguments to path.read_cdf UI: what now defaults to NULL and defaults are coded into downstream read_andi_chrom and read_andi_ms functions.read_chroms when verbose is TRUE.reticulate::configure_environment).data.table as an option for format_out.read_shimadzu_lcd by dealing with twos-complements more sensibly.path_out when using 'ThermoRawFileParser' and 'OpenChrom' parsers.read_chroms if path_out does not exist.mdf files lacking null bytes after the file header.read_chroms.export_csv function to label first column for wide-format chromatograms..ch) files through read_chemstation_ch.Agilent Chemstation files due to typo.ChemStation CSV files on (some) mac machines by specifying little-endian format in call to read.csv.read_shimadzu_lcd_2d to more accurately reflect file structure.scale argument to read_chemstation_ch and read_shimadzu_lcd to
toggle scaling of chromatograms resolving (#30).read_shimadzu_lcd function to correctly determine the number of blocks in the "Shimadzu" LCD PDA stream (thanks to kco-hereon).export_cdfs function to permit conversion of files lacking metadata..cdf) files.read_chemstation_ch parser to correctly read "Mustang Chemstation" 179 files with 8-byte encoding.read_shimadzu function and added support for new types of chromatograms (e.g. status, uv and total ion chromatograms). Added support for reading multiple types of chromatograms at once.read_shimadzu.write_cdf and added additional arguments (lambda and force) for greater control by users.read_waters_raw).collapse argument to call_rainbow and to collapse superfluous lists.... argument to read_chroms for supplying additional arguments to parsers.read_chroms for reading mzxml files with RaMS.precision argument to call_rainbow to control number of digits "mz" values are rounded to. (Also changed default behavior so values are rounded to one decimal by default).read_shimadzu_lcd on Windows due to issue with passing escaped paths to Python.pbapply package. (Note: The pbapply package must be manually installed to enable parallel processing).read_chemstation_uv function)..dx) files (through read_agilentdx function)..lcd files through the read_shimadzu_lcd function. Only the PDA stream (not MS) is currently supported.read_peaklist function for reading peak lists. Currently 'Agilent ChemStation' and 'Shimadzu ASCII' formats are supported.verbose argument to control console output for external parsers ('OpenChrom' and 'ThermoRawFileParser').read_chroms.read_thermoraw function to simplify paths.thermoraw and openchrom parsers now use a proper temp directory if an export directory is not specified through the path_out argument.reshape_chroms, speeding up conversion from wide to long format.openchrom export format to mzml.what == "peak_table" and read_metadata == TRUE).format_out == "matrix".data_format attributes in 'Waters ARW' and 'Chromeleon' parsers.chemstation_ch parser (version 130) (#17).arw) PDA files..ch) files (version 30).read_chromeleon to better deal with comma decimal separators in metadata.read_chromeleon to deal with more datetime formats.read_chromeleon to deal with unicode microliters.read_chemstation_ch.cdf files through the read_cdf function..mdf files through the read_mdf function.chemstation_csv (utf-16) and ANDI chrom cdf files through read_chroms.read_chroms when providing direct paths to files (i.e. when find_files == FALSE).read_varian_peaklist function for reading peak lists from 'Varian MS Workstation'.wide and long data_format options for 2D data, such that the wide format option writes retention times as rownames of the matrix or data.frame. while the long format writes retention times as the first column of the object.configure_openchrom for better discovery of 'OpenChrom' path and added path argument for directly specifying the path to 'OpenChrom'.source_file field to track
data origin.fs package for parsing paths, eliminating buggy check_paths function..uv) files (version 131) through
the read_chemstation_uv function.extract_metadata function for extracting metadata from a list of chromatograms
and returning it as a data.frame or tibble.progress_bar option in read_chroms.reshape_chroms and reshape_chrom to allow switching between "wide" and "long" formats.read_mzml.read_chroms.read_chroms.POSIXct format..ch) files (version 130).read_chemstation_fid function to read_chemstation_ch.read_chroms.format_data argument to data_format to select wide or long format.mzML files with RaMS..D) and "Waters" (.raw) files with rainbow.data_format option available consistently for choosing wide or long format..ch) data (versions 81, 179 & 181).call_openchrom so that it can actually find 'OpenChrom' path.call_openchrom to allow 'animl' as valid export_format.configure_aston function for configuration of Aston parsers and fixed issues with configuration of Aston.read_shimadzu function for parsing 'Shimadzu' ascii files.read_chromeleon function for parsing 'Chromeleon' ascii files.read_thermoraw function to convert 'Thermo Raw' files by calling the 'ThermoRawFileParser'.read_mzml function to extract UV data from mzML files using mzR.call_entab function for calling Entab parsers.call_openchrom to call OpenChrom parsers through the command-line interface.(All of the new functions described above can be called from the read_chroms function by setting the format_in and parser arguments).
read_metadata = TRUE in read_chroms.read_chroms will now automatically assign a parser if the parser isn't specified.read_chroms will throw more informative errors for mismatch between format_in and parser arguments.read_chroms will try to automatically determine if files or directories are being provided.Dots were replaced with underscores in all arguments to read_chroms for internal syntactical consistency across the package. Thus:
R.format to format_out.format.out to export_format.NEWS.md file to track changes to the package.