Извлеките текстовые данные из файлов

В этом примере показано, как извлечь текстовые данные из текста, HTML, Microsoft® Word, PDF, CSV и файлы Microsoft Excel® и импортировать его в MATLAB® для анализа.

Обычно, самый легкий способ импортировать текстовые данные в MATLAB состоит в том, чтобы использовать extractFileText функция. Эта функция извлекает текстовые данные из текста, PDF, HTML и файлов Microsoft Word. Чтобы импортировать текст из CSV и файлов Microsoft Excel, используйте readtable. Чтобы извлечь текст из кода HTML, используйте extractHTMLText. Чтобы считать данные из форм PDF, используйте readPDFFormData.

Текстовый файл

Извлеките текст из sonnets.txt использование extractFileText. Файл sonnets.txt содержит сонеты Шекспира в простом тексте.

filename = "sonnets.txt";
str = extractFileText(filename);

Просмотрите первый сонет путем извлечения текста между этими двумя заголовками "I"и "II".

start = " I" + newline;
fin = " II";
sonnet1 = extractBetween(str,start,fin)
sonnet1 = 
    "
       From fairest creatures we desire increase,
       That thereby beauty's rose might never die,
       But as the riper should by time decease,
       His tender heir might bear his memory:
       But thou, contracted to thine own bright eyes,
       Feed'st thy light's flame with self-substantial fuel,
       Making a famine where abundance lies,
       Thy self thy foe, to thy sweet self too cruel:
       Thou that art now the world's fresh ornament,
       And only herald to the gaudy spring,
       Within thine own bud buriest thy content,
       And tender churl mak'st waste in niggarding:
         Pity the world, or else this glutton be,
         To eat the world's due, by the grave and thee.
     
      "

Документ Microsoft Word

Извлеките текст из sonnets.docx использование extractFileText. Файл exampleSonnets.docx содержит сонеты Шекспира в документе Microsoft Word.

filename = "exampleSonnets.docx";
str = extractFileText(filename);

Просмотрите второй сонет путем извлечения текста между этими двумя заголовками "II"и "III".

start = " II" + newline;
fin = " III";
sonnet2 = extractBetween(str,start,fin)
sonnet2 = 
    "
       When forty winters shall besiege thy brow,
     
       And dig deep trenches in thy beauty's field,
     
       Thy youth's proud livery so gazed on now,
     
       Will be a tatter'd weed of small worth held:
     
       Then being asked, where all thy beauty lies,
     
       Where all the treasure of thy lusty days;
     
       To say, within thine own deep sunken eyes,
     
       Were an all-eating shame, and thriftless praise.
     
       How much more praise deserv'd thy beauty's use,
     
       If thou couldst answer 'This fair child of mine
     
       Shall sum my count, and make my old excuse,'
     
       Proving his beauty by succession thine!
     
         This were to be new made when thou art old,
     
         And see thy blood warm when thou feel'st it cold.
     
      "

Пример документ Microsoft Word использует два символа новой строки между каждой линией. Чтобы заменить эти символы на один символ новой строки, используйте replace функция.

sonnet2 = replace(sonnet2,[newline newline],newline)
sonnet2 = 
    "
       When forty winters shall besiege thy brow,
       And dig deep trenches in thy beauty's field,
       Thy youth's proud livery so gazed on now,
       Will be a tatter'd weed of small worth held:
       Then being asked, where all thy beauty lies,
       Where all the treasure of thy lusty days;
       To say, within thine own deep sunken eyes,
       Were an all-eating shame, and thriftless praise.
       How much more praise deserv'd thy beauty's use,
       If thou couldst answer 'This fair child of mine
       Shall sum my count, and make my old excuse,'
       Proving his beauty by succession thine!
         This were to be new made when thou art old,
         And see thy blood warm when thou feel'st it cold.
      "

Файлы PDF

Извлеките текст из документов в формате PDF и данные из форм PDF.

Документ в формате PDF

Извлеките текст из sonnets.pdf использование extractFileText. Файл exampleSonnets.pdf содержит сонеты Шекспира в PDF.

filename = "exampleSonnets.pdf";
str = extractFileText(filename);

Просмотрите третий сонет путем извлечения текста между этими двумя заголовками "III"и "IV". Этот PDF имеет пробел перед каждым символом новой строки.

start = " III " + newline;
fin = "IV";
sonnet3 = extractBetween(str,start,fin)
sonnet3 = 
    " 
       Look in thy glass and tell the face thou viewest 
       Now is the time that face should form another; 
       Whose fresh repair if now thou not renewest, 
       Thou dost beguile the world, unbless some mother. 
       For where is she so fair whose unear'd womb 
       Disdains the tillage of thy husbandry? 
       Or who is he so fond will be the tomb, 
       Of his self-love to stop posterity? 
       Thou art thy mother's glass and she in thee 
       Calls back the lovely April of her prime; 
       So thou through windows of thine age shalt see, 
       Despite of wrinkles this thy golden time. 
         But if thou live, remember'd not to be, 
         Die single and thine image dies with thee. 
     
      
       "

Форма PDF

Чтобы считать текстовые данные из форм PDF, используйте readPDFFormData. Функция возвращает struct, содержащий данные из полей формы PDF.

filename = "weatherReportForm1.pdf";
data = readPDFFormData(filename)
data = struct with fields:
         event_type: "Thunderstorm Wind"
    event_narrative: "Large tree down between Plantersville and Nettleton."

HTML

Извлеките текст из файлов HTML, кода HTML и сети.

Файл HTML

Чтобы извлечь текстовые данные из сохраненного файла HTML, используйте extractFileText.

filename = "exampleSonnets.html";
str = extractFileText(filename);

Просмотрите дальше сонет путем извлечения текста между этими двумя заголовками "IV" и "V".

start = newline + "IV" + newline;
fin = newline + "V" + newline;
sonnet4 = extractBetween(str,start,fin)
sonnet4 = 
    "
     Unthrifty loveliness, why dost thou spend
      Upon thy self thy beauty's legacy?
      Nature's bequest gives nothing, but doth lend,
      And being frank she lends to those are free:
      Then, beauteous niggard, why dost thou abuse
      The bounteous largess given thee to give?
      Profitless usurer, why dost thou use
      So great a sum of sums, yet canst not live?
      For having traffic with thy self alone,
      Thou of thy self thy sweet self dost deceive:
      Then how when nature calls thee to be gone,
      What acceptable audit canst thou leave?
      Thy unused beauty must be tombed with thee,
      Which, used, lives th' executor to be.
     "

КОД HTML

Чтобы извлечь текстовые данные из кода HTML строки, содержащей, используйте extractHTMLText.

code = "<html><body><h1>THE SONNETS</h1><p>by William Shakespeare</p></body></html>";
str = extractHTMLText(code)
str = 
    "THE SONNETS
     
     by William Shakespeare"

С сети

Чтобы извлечь текстовые данные из веб-страницы, сначала считайте код HTML с помощью webread, и затем используйте extractHTMLText.

url = "https://www.mathworks.com/help/textanalytics";
code = webread(url);
str = extractHTMLText(code)
str = 
    'Text Analytics Toolbox™ provides algorithms and visualizations for preprocessing, analyzing, and modeling text data. Models created with the toolbox can be used in applications such as sentiment analysis, predictive maintenance, and topic modeling.
     
     Text Analytics Toolbox includes tools for processing raw text from sources such as equipment logs, news feeds, surveys, operator reports, and social media. You can extract text from popular file formats, preprocess raw text, extract individual words, convert text into numerical representations, and build statistical models.
     
     Using machine learning techniques such as LSA, LDA, and word embeddings, you can find clusters and create features from high-dimensional text datasets. Features created with Text Analytics Toolbox can be combined with features from other data sources to build machine learning models that take advantage of textual, numeric, and other types of data.'

Проанализируйте КОД HTML

Чтобы найти конкретные элементы кода HTML, проанализируйте код с помощью htmlTree и используйте findElement. Проанализируйте код HTML и найдите все гиперссылки. Гиперссылки являются узлами с именем элемента "A".

tree = htmlTree(code);
selector = "A";
subtrees = findElement(tree,selector);

Просмотрите первые 10 поддеревьев и извлеките текст с помощью extractHTMLText.

subtrees(1:10)
ans = 
  10×1 htmlTree:

    <A class="svg_link navbar-brand" href="https://www.mathworks.com?s_tid=gn_logo"><IMG alt="MathWorks" class="mw_logo" src="/images/responsive/global/pic-header-mathworks-logo.svg"/></A>
    <A href="https://www.mathworks.com/products.html?s_tid=gn_ps">Products</A>
    <A href="https://www.mathworks.com/solutions.html?s_tid=gn_sol">Solutions</A>
    <A href="https://www.mathworks.com/academia.html?s_tid=gn_acad">Academia</A>
    <A href="https://www.mathworks.com/support.html?s_tid=gn_supp">Support</A>
    <A href="https://www.mathworks.com/matlabcentral/?s_tid=gn_mlc">Community</A>
    <A href="https://www.mathworks.com/company/events.html?s_tid=gn_ev">Events</A>
    <A href="https://www.mathworks.com/company/aboutus/contact_us.html?s_tid=gn_cntus">Contact Us</A>
    <A href="https://www.mathworks.com/products/get-matlab.html?s_tid=gn_getml">Get MATLAB</A>
    <A class="svg_link pull-left" href="https://www.mathworks.com?s_tid=gn_logo"><IMG alt="MathWorks" class="mw_logo" src="/images/responsive/global/pic-header-mathworks-logo.svg"/></A>

str = extractHTMLText(subtrees);

Просмотрите извлеченный текст первых 10 гиперссылок.

str(1:10)
ans = 10×1 string
    ""
    "Products"
    "Solutions"
    "Academia"
    "Support"
    "Community"
    "Events"
    "Contact Us"
    "Get MATLAB"
    ""

Чтобы получить цели ссылки, используйте getAttributes и задайте атрибут "href" (ссылка гиперссылки). Получите цели ссылки первых 10 поддеревьев.

attr = "href";
str = getAttribute(subtrees(1:10),attr)
str = 10×1 string
    "https://www.mathworks.com?s_tid=gn_logo"
    "https://www.mathworks.com/products.html?s_tid=gn_ps"
    "https://www.mathworks.com/solutions.html?s_tid=gn_sol"
    "https://www.mathworks.com/academia.html?s_tid=gn_acad"
    "https://www.mathworks.com/support.html?s_tid=gn_supp"
    "https://www.mathworks.com/matlabcentral/?s_tid=gn_mlc"
    "https://www.mathworks.com/company/events.html?s_tid=gn_ev"
    "https://www.mathworks.com/company/aboutus/contact_us.html?s_tid=gn_cntus"
    "https://www.mathworks.com/products/get-matlab.html?s_tid=gn_getml"
    "https://www.mathworks.com?s_tid=gn_logo"

CSV и файлы Microsoft Excel

Чтобы извлечь текстовые данные из CSV и файлов Microsoft Excel, используйте readtable и извлеките текстовые данные из таблицы, которую они возвращают.

Извлеките табличные данные из factoryReposts.csv использование readtable функционируйте и просмотрите первые несколько строк таблицы.

T = readtable('factoryReports.csv','TextType','string');
head(T)
ans=8×5 table
                                 Description                                       Category          Urgency          Resolution         Cost
    _____________________________________________________________________    ____________________    ________    ____________________    ____

    "The materials get jammed in the mixer."                                 "Mechanical Failure"    "Medium"    "Readjust Machine"       35 
    "The controller software keeps crashing."                                "Software Failure"      "Low"       "Update Firmware"       145 
    "Some drips of liquid are appearing underneath the assembler."           "Leak"                  "Medium"    "Add to Watch List"      71 
    "There are some high pitched electrical sounds emitted by the mixer."    "Electronic Failure"    "Low"       "Add to Watch List"      78 
    "Severe coolant leak underneath mixer."                                  "Leak"                  "High"      "Replace Components"    460 
    "The mixer is smoking during operation."                                 "Electronic Failure"    "High"      "Replace Components"    438 
    "The scanner spools are sometimes jamming."                              "Mechanical Failure"    "Low"       "Add to Watch List"      49 
    "Some liquid pooling underneath output of assembler."                    "Leak"                  "High"      "Replace Components"    410 

Извлеките текстовые данные из event_narrative столбец и представление первые несколько строк.

str = T.Description;
str(1:10)
ans = 10×1 string
    "The materials get jammed in the mixer."
    "The controller software keeps crashing."
    "Some drips of liquid are appearing underneath the assembler."
    "There are some high pitched electrical sounds emitted by the mixer."
    "Severe coolant leak underneath mixer."
    "The mixer is smoking during operation."
    "The scanner spools are sometimes jamming."
    "Some liquid pooling underneath output of assembler."
    "The products occasionally leave the scanner cracked."
    "The sorter motor keeps getting jammed."

Извлеките текст из нескольких файлов

Если ваши текстовые данные содержатся в нескольких файлах в папке, то можно импортировать текстовые данные в MATLAB с помощью datastore файла.

Создайте datastore файла для текстовых файлов сонета в качестве примера. Файлы в качестве примера называют "exampleSonnetN.txt", где N количество сонета. Задайте имя файла с помощью подстановочного знака "*", чтобы найти все имена файлов этой структуры. Чтобы задать чтение функционируют, чтобы быть extractFileText, введите эту функцию к fileDatastore использование указателя на функцию.

location = fullfile(matlabroot,"examples","textanalytics","data","exampleSonnet*.txt");
fds = fileDatastore(location,'ReadFcn',@extractFileText)
fds = 
  FileDatastore with properties:

                       Files: {
                              ' ...\matlab\examples\textanalytics\data\exampleSonnet1.txt';
                              ' ...\matlab\examples\textanalytics\data\exampleSonnet2.txt';
                              ' ...\matlab\examples\textanalytics\data\exampleSonnet3.txt'
                               ... and 2 more
                              }
                     Folders: {
                              ' ...\matlab\examples\textanalytics\data'
                              }
                 UniformRead: 0
                    ReadMode: 'file'
                   BlockSize: Inf
                  PreviewFcn: @extractFileText
      SupportedOutputFormats: ["txt"    "csv"    "xlsx"    "xls"    "parquet"    "parq"    "png"    "jpg"    "jpeg"    "tif"    "tiff"    "wav"    "flac"    "ogg"    "mp4"    "m4a"]
                     ReadFcn: @extractFileText
    AlternateFileSystemRoots: {}

Цикл по файлам в datastore и считал каждый текстовый файл.

str = [];
while hasdata(fds)
    textData = read(fds);
    str = [str; textData];
end

Просмотрите извлеченный текст.

str
str = 5×1 string
    "  From fairest creatures we desire increase,↵  That thereby beauty's rose might never die,↵  But as the riper should by time decease,↵  His tender heir might bear his memory:↵  But thou, contracted to thine own bright eyes,↵  Feed'st thy light's flame with self-substantial fuel,↵  Making a famine where abundance lies,↵  Thy self thy foe, to thy sweet self too cruel:↵  Thou that art now the world's fresh ornament,↵  And only herald to the gaudy spring,↵  Within thine own bud buriest thy content,↵  And tender churl mak'st waste in niggarding:↵    Pity the world, or else this glutton be,↵    To eat the world's due, by the grave and thee."
    "  When forty winters shall besiege thy brow,↵  And dig deep trenches in thy beauty's field,↵  Thy youth's proud livery so gazed on now,↵  Will be a tatter'd weed of small worth held:↵  Then being asked, where all thy beauty lies,↵  Where all the treasure of thy lusty days;↵  To say, within thine own deep sunken eyes,↵  Were an all-eating shame, and thriftless praise.↵  How much more praise deserv'd thy beauty's use,↵  If thou couldst answer 'This fair child of mine↵  Shall sum my count, and make my old excuse,'↵  Proving his beauty by succession thine!↵    This were to be new made when thou art old,↵    And see thy blood warm when thou feel'st it cold."
    "  Look in thy glass and tell the face thou viewest↵  Now is the time that face should form another;↵  Whose fresh repair if now thou not renewest,↵  Thou dost beguile the world, unbless some mother.↵  For where is she so fair whose unear'd womb↵  Disdains the tillage of thy husbandry?↵  Or who is he so fond will be the tomb,↵  Of his self-love to stop posterity?↵  Thou art thy mother's glass and she in thee↵  Calls back the lovely April of her prime;↵  So thou through windows of thine age shalt see,↵  Despite of wrinkles this thy golden time.↵    But if thou live, remember'd not to be,↵    Die single and thine image dies with thee."
    "  Unthrifty loveliness, why dost thou spend↵  Upon thy self thy beauty's legacy?↵  Nature's bequest gives nothing, but doth lend,↵  And being frank she lends to those are free:↵  Then, beauteous niggard, why dost thou abuse↵  The bounteous largess given thee to give?↵  Profitless usurer, why dost thou use↵  So great a sum of sums, yet canst not live?↵  For having traffic with thy self alone,↵  Thou of thy self thy sweet self dost deceive:↵  Then how when nature calls thee to be gone,↵  What acceptable audit canst thou leave?↵    Thy unused beauty must be tombed with thee,↵    Which, used, lives th' executor to be."
    "from fairest creatures we desire increase that thereby beautys rose might never die but as the riper should by time decease his tender heir might bear his memory but thou contracted to thine own bright eyes feedst thy lights flame with selfsubstantial fuel making a famine where abundance lies thy self thy foe to thy sweet self too cruel thou that art now the worlds fresh ornament and only herald to the gaudy spring within thine own bud buriest thy content and tender churl makst waste in niggarding pity the world or else this glutton be to eat the worlds due by the grave and thee↵when forty winters shall besiege thy brow and dig deep trenches in thy beautys field thy youths proud livery so gazed on now will be a tatterd weed of small worth held then being asked where all thy beauty lies where all the treasure of thy lusty days to say within thine own deep sunken eyes were an alleating shame and thriftless praise how much more praise deservd thy beautys use if thou couldst answer this fair child of mine shall sum my count and make my old excuse proving his beauty by succession thine this were to be new made when thou art old and see thy blood warm when thou feelst it cold↵look in thy glass and tell the face thou viewest now is the time that face should form another whose fresh repair if now thou not renewest thou dost beguile the world unbless some mother for where is she so fair whose uneard womb disdains the tillage of thy husbandry or who is he so fond will be the tomb of his selflove to stop posterity thou art thy mothers glass and she in thee calls back the lovely april of her prime so thou through windows of thine age shalt see despite of wrinkles this thy golden time but if thou live rememberd not to be die single and thine image dies with thee↵unthrifty loveliness why dost thou spend upon thy self thy beautys legacy natures bequest gives nothing but doth lend and being frank she lends to those are free then beauteous niggard why dost thou abuse the bounteous largess given thee to give profitless usurer why dost thou use so great a sum of sums yet canst not live for having traffic with thy self alone thou of thy self thy sweet self dost deceive then how when nature calls thee to be gone what acceptable audit canst thou leave thy unused beauty must be tombed with thee which used lives th executor to be↵those hours that with gentle work did frame the lovely gaze where every eye doth dwell will play the tyrants to the very same and that unfair which fairly doth excel for neverresting time leads summer on to hideous winter and confounds him there sap checked with frost and lusty leaves quite gone beauty oersnowed and bareness every where then were not summers distillation left a liquid prisoner pent in walls of glass beautys effect with beauty were bereft nor it nor no remembrance what it was but flowers distilld though they with winter meet leese but their show their substance still lives sweet↵then let not winters ragged hand deface in thee thy summer ere thou be distilld make sweet some vial treasure thou some place with beautys treasure ere it be selfkilld that use is not forbidden usury which happies those that pay the willing loan thats for thy self to breed another thee or ten times happier be it ten for one ten times thy self were happier than thou art if ten of thine ten times refigurd thee then what could death do if thou shouldst depart leaving thee living in posterity be not selfwilld for thou art much too fair to be deaths conquest and make worms thine heir↵lo in the orient when the gracious light lifts up his burning head each under eye doth homage to his newappearing sight serving with looks his sacred majesty and having climbd the steepup heavenly hill resembling strong youth in his middle age yet mortal looks adore his beauty still attending on his golden pilgrimage but when from highmost pitch with weary car like feeble age he reeleth from the day the eyes fore duteous now converted are from his low tract and look another way so thou thyself outgoing in thy noon unlookd on diest unless thou get a son↵music to hear why hearst thou music sadly sweets with sweets war not joy delights in joy why lovst thou that which thou receivst not gladly or else receivst with pleasure thine annoy if the true concord of welltuned sounds by unions married do offend thine ear they do but sweetly chide thee who confounds in singleness the parts that thou shouldst bear mark how one string sweet husband to another strikes each in each by mutual ordering resembling sire and child and happy mother who all in one one pleasing note do sing whose speechless song being many seeming one sings this to thee thou single wilt prove none↵is it for fear to wet a widows eye that thou consumst thy self in single life ah if thou issueless shalt hap to die the world will wail thee like a makeless wife the world will be thy widow and still weep that thou no form of thee hast left behind when every private widow well may keep by childrens eyes her husbands shape in mind look what an unthrift in the world doth spend shifts but his place for still the world enjoys it but beautys waste hath in the world an end and kept unused the user so destroys it no love toward others in that bosom sits that on himself such murdrous shame commits↵for shame deny that thou bearst love to any who for thy self art so unprovident grant if thou wilt thou art belovd of many but that thou none lovst is most evident for thou art so possessd with murderous hate that gainst thy self thou stickst not to conspire seeking that beauteous roof to ruinate which to repair should be thy chief desire o change thy thought that i may change my mind shall hate be fairer lodgd than gentle love be as thy presence is gracious and kind or to thyself at least kindhearted prove make thee another self for love of me that beauty still may live in thine or thee↵as fast as thou shalt wane so fast thou growst in one of thine from that which thou departest and that fresh blood which youngly thou bestowst thou mayst call thine when thou from youth convertest herein lives wisdom beauty and increase without this folly age and cold decay if all were minded so the times should cease and threescore year would make the world away let those whom nature hath not made for store harsh featureless and rude barrenly perish look whom she best endowd she gave thee more which bounteous gift thou shouldst in bounty cherish she carvd thee for her seal and meant thereby thou shouldst print more not let that copy die↵when i do count the clock that tells the time and see the brave day sunk in hideous night when i behold the violet past prime and sable curls all silvered oer with white when lofty trees i see barren of leaves which erst from heat did canopy the herd and summers green all girded up in sheaves borne on the bier with white and bristly beard then of thy beauty do i question make that thou among the wastes of time must go since sweets and beauties do themselves forsake and die as fast as they see others grow and nothing gainst times scythe can make defence save breed to brave him when he takes thee hence↵o that you were your self but love you are no longer yours than you your self here live against this coming end you should prepare and your sweet semblance to some other give so should that beauty which you hold in lease find no determination then you were yourself again after yourselfs decease when your sweet issue your sweet form should bear who lets so fair a house fall to decay which husbandry in honour might uphold against the stormy gusts of winters day and barren rage of deaths eternal cold o none but unthrifts dear my love you know you had a father let your son say so↵not from the stars do i my judgement pluck and yet methinks i have astronomy but not to tell of good or evil luck of plagues of dearths or seasons quality nor can i fortune to brief minutes tell pointing to each his thunder rain and wind or say with princes if it shall go well by oft predict that i in heaven find but from thine eyes my knowledge i derive and constant stars in them i read such art as truth and beauty shall together thrive if from thyself to store thou wouldst convert or else of thee this i prognosticate thy end is truths and beautys doom and date↵when i consider every thing that grows holds in perfection but a little moment that this huge stage presenteth nought but shows whereon the stars in secret influence comment when i perceive that men as plants increase cheered and checked even by the selfsame sky vaunt in their youthful sap at height decrease and wear their brave state out of memory then the conceit of this inconstant stay sets you most rich in youth before my sight where wasteful time debateth with decay to change your day of youth to sullied night and all in war with time for love of you as he takes from you i engraft you new↵but wherefore do not you a mightier way make war upon this bloody tyrant time and fortify your self in your decay with means more blessed than my barren rhyme now stand you on the top of happy hours and many maiden gardens yet unset with virtuous wish would bear you living flowers much liker than your painted counterfeit so should the lines of life that life repair which this times pencil or my pupil pen neither in inward worth nor outward fair can make you live your self in eyes of men to give away yourself keeps yourself still and you must live drawn by your own sweet skill↵who will believe my verse in time to come if it were filld with your most high deserts though yet heaven knows it is but as a tomb which hides your life and shows not half your parts if i could write the beauty of your eyes and in fresh numbers number all your graces the age to come would say this poet lies such heavenly touches neer touchd earthly faces so should my papers yellowd with their age be scornd like old men of less truth than tongue and your true rights be termd a poets rage and stretched me…"

Смотрите также

| | |

Похожие темы