- Corpora - ELRA lists

[CfP] DBpedia Day @ SEMANTiCS on Sep 20
by DBpedia 11 Aug '23

11 Aug '23

1 0

Postdoc position in Cambridge (deadline 15/9)
by A. Vlachos 10 Aug '23

10 Aug '23

Dear all, We are looking for a Research Assistant/Associate to work on the project Automated Verification of Textual Claims (AVeriTeC), an ERC project led by Prof. Andreas Vlachos at the University of Cambridge starting in January 2024 or as soon as possible thereafter. The position is for two years. The successful candidate will be based in the Natural Language and Information Processing group (http://www.cl.cam.ac.uk/research/nl/) at the Department of Computer Science and Technology. The project focuses on developing approaches enabling the verification of highly complex claims, which require multiple pieces of evidence. Special focus will be paid to accompanying the verdicts with suitable justifications. Candidates will have completed a Ph.D. (or be close to completing it) in a relevant field such as NLP, Information Retrieval, Artificial Intelligence or Machine Learning and be able to demonstrate a strong track record of independent research and high-quality publications. Essential skills include excellent programming (Python), NLP techniques, Machine Learning, and proven communication skills. Appointment at research associate level is dependent on having a PhD or having equivalent skills and experience through non-academic routes. Where a PhD has yet to be awarded appointment will initially be made as a research assistant and amended to research associate when the PhD is awarded. Enquiries concerning this position should be directed to Prof. Andreas Vlachos (av308(a)cam.ac.uk), and applicants are encouraged to contact him regarding the position. Apply using this link: https://www.jobs.cam.ac.uk/job/42366/ Thanks, Andreas

1 0

Free event: 22 September - Business and Professional Communication
by Brezina, Vaclav 10 Aug '23

10 Aug '23

Dear all, We are organising a free hybrid event (online and in person): Language Data Analysis for Business and Professional Communication. It will take place on 22 September 2023 10:00 - 15:30 UK time. More details and registration: https://cass.lancs.ac.uk/mycalendar-events/?event_id1=3470 The ESRC Centre for Corpus Approaches to Social Science, Lancaster University offers a practical training workshop focused on computational analysis of language data for businesses and professional organisations and anyone interested in communication in professional contexts. The data includes social media, newspapers, business reports, marketing materials and other data sources. The workshop will introduce a new software tool #LancsBox X<https://lancsbox.lancs.ac.uk/> developed at Lancaster University, which can analyse and visualise large amounts of language data (millions and billions of words). Practical examples of uses of #LancsBox X (case studies) will be provided. Best, Vaclav Professor Vaclav Brezina Professor in Corpus Linguistics Department of Linguistics and English Language ESRC Centre for Corpus Approaches to Social Science Faculty of Arts and Social Sciences, Lancaster University Lancaster, LA1 4YD Office: County South, room C05 T: +44 (0)1524 510828 [cid:02038f34-9006-4e55-866f-0bc6e77964ba]@vaclavbrezina [cid:30848364-59cf-4561-a71f-9c3e4ac4519c]<http://www.lancaster.ac.uk/arts-and-social-sciences/about-us/people/vaclav-…>

1 0

[FinNLP@IJCNLP-AACL-2023] Call for Paper & ML-ESG-2 Shared Task
by CHUNG-CHI CHEN 10 Aug '23

10 Aug '23

Dear Colleagues, I'm delighted to share that The 6th Workshop on Financial Technology and Natural Language Processing (FinNLP) will be collocated with IJCNLP-AACL-2023 (http://www.ijcnlp-aacl2023.org/) in Bali, Indonesia (November 1–4). The submission due for the main track paper is *Sep. 8th, 2023*. For more details, please refer to the website: https://sites.google.com/nlg.csie.ntu.edu.tw/finnlp2023/home We continue the *ML-ESG shared task* in FinNLP, and share new labels related to Impact Type Identification. You can also get more details about the shared task on the FinNLP website. The Registration Form is ready: https://forms.gle/j6gL5jy1upq5LrKY9 Participants in previous FinNLP share various insights into NLP in FinTech applications. Please refer to past years' proceedings for more details: https://aclanthology.org/venues/finnlp/ Feel free to let us know if you have any questions. Looking forward to seeing you at the 6th FinNLP. Best Regards, Chung-Chi --- 陳重吉 (Chung-Chi Chen), Ph.D. Researcher Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology, Japan E-mail: c.c.chen(a)acm.org <cjchen(a)nlg.csie.ntu.edu.tw> Website: http://cjchen.nlpfin.com

1 0

Arizona Linguistic Circle 17 -Collaborative work in Linguistics: Communities and Societies
by heather froehlich 09 Aug '23

09 Aug '23

Dear all, Forwarding on behalf of the Arizona Linguistics Circle 17 (ALC 17) <https://sites.google.com/view/arizonalinguisticscircle17/home?authuser=3> committee: We are happy to announce the 17th annual ALC conference which will take place Halloweekend🎃 (October 27 to 29, 2023) at the University of Arizona in Tucson, Arizona. The theme for the conference is "Collaborative work in Linguistics: Communities and Societies." We welcome graduate students, undergraduate students, and language workers in communities to submit a proposal. In the coming weeks we will announce our invited plenary speakers for ALC 17. Please see below for the Call for Papers which can also be found via this <https://easychair.org/cfp/ALC17>EasyChair <https://easychair.org/cfp/ALC17> link. Arizona Linguistics Circle 17 Collaborative work in Linguistics: Communities and Societies University of Arizona Tucson, Arizona October 27 to 29, 2023 Keynote Speakers: TBA Call for Papers Submissions We are pleased to invite talk proposals for Arizona Linguistics Circle 17 (ALC 17) <https://sites.google.com/view/arizonalinguisticscircle17/home?authuser=3>. ALC 17 is an annual graduate student-run conference held at the University of Arizona. Our goal is to foster a deeper appreciation for linguistics while providing a healthy environment for academic discussion, especially as it concerns graduate student research. The theme of this year’s conference is Collaborative work in Linguistics, that is the ways in which we have worked to adapt to the unique challenges of a pandemic, found new ways to conduct research remotely, and made ethical considerations in using different modalities. However, abstracts from all areas of linguistics are welcome and encouraged. This year, we plan to conduct a hybrid conference, with both in-person and remote aspects (see Covid contingency) Abstract Guidelines: - Abstract may not exceed 500 words (not including keywords, references, figures, and tableaux) - Presentations will consist of 20-minute talks with 10-minute Q&A - Authors are limited to one individual and one joint abstract (not including workshop submissions) - Only anonymized submissions will be accepted - Abstract submission deadline Friday, September 1, 2023 Abstract submission is via EasyChair <https://easychair.org/cfp/ALC17>. For questions regarding abstract submission, please contact the abstract review manager at azlingcircle17(a)gmail.com. Notification of acceptance will be sent in late August. Proceedings Presenters will be invited to submit their paper for publication in Coyote Papers <https://coyotepapers.sbs.arizona.edu/>, the conference proceedings for ALC. Call for Workshop Submissions We are also inviting proposals for one- to two-hour long workshops on the theme of Collaborative work in Linguistics within the realm of experimentation, fieldwork, and data collection. Example workshops include, but are not limited to, improving sound quality in remote fieldwork, getting the most from online crowdsourcing or experimentation platforms, setting up remote speech collection experiments, and more. Workshop submissions from students are especially welcome. Workshop abstracts may not exceed 500 words and must include a title, detailed description (topic, format, length, and content), technical prerequisites (hardware, software, or platforms), applicant's areas of expertise, and why the workshop would be of interest to ALC attendees. Workshop abstracts are submitted on EasyChair and do not count towards the limit on paper abstract submissions. Authors of accepted workshops will be offered modest honoraria. Covid contingency Our plan is to enjoy an in-person conference. However, we might consider a hybrid conference, such that presenters will have the option to present in-person or remotely. This way, both domestic and international presenters will have the opportunity to deliver their presentations regardless of the Covid situation or vaccine availability of their location. We will also have a contingency plan to move the conference to a virtual format if Arizona is undergoing a significant Covid outbreak at the time of the conference. Please let us know if you have any questions via email: azlingcircle17(a)gmail.com. We look forward to sharing more information with you over the coming months as we prepare for ALC 17. Attentively, Jesús E. González Franco (ALC 17 co-chair) -- Dr Heather Froehlich w // http://hfroehli.ch t // @heatherfro

1 0

3rd Artificial Intelligence for Industry, Science, and Society: Geneva, 11 to 15 September 2023
by Douglas Teodoro 09 Aug '23

09 Aug '23

Hello, We are pleased to announce that the University of Geneva will be hosting the 3rd Symposium on Artificial Intelligence for Industry, Science, and Society (AI2S2) from 11 to 15 September 2023, at Campus Biotech, Geneva. *Both in-person and remote-only registration are free*; registration closes soon, thus we encourage you to register ASAP if you intend to participate. We will be welcoming several renowned experts from each of the event pillars (industry, science, and society), and have an exciting schedule covering a wide variety of topics of interest to the community, including keynote presentations from the following experts: - *Alistair Knott*, Professor of AI; Victoria University of Wellington - *Andrew Wyckoff*, Former Director for Science, Technology, and Innovation; OECD - *Ger Janssen*, Principal Scientist for AI & Data Science & Digital Twin; Philips - *I**nma Martinez*, Chair of the Multi-Stakeholder Committee; GPAI - *Juha Heikkilä*, International Advisor for AI; European Commission - *Laure Soulier*, Professor HDR-ISIR Lab; Sorbonne University - *Maria Girone*, Head of OpenLab; CERN - *Michael Bronstein*, DeepMind Professor of AI; Oxford University - *Philippe Limantour*, Chief Technology and Cybersecurity Officer; Microsoft France - *Stephen MacFeely*, Director of Data and Analytics; World Health Organisation You can find more information about the event at the following links: - Main event website: https://ai2s2.org/2023/ - Event timetable/agenda: https://ai2s2.org/2023/event/ - Registration (in person): link <https://indico.cern.ch/event/1288077/page/29800-general-registration> - Registration (remote only): link <https://indico.cern.ch/event/1288077/page/29802-remote-registration> We hope to see many of you next month! Best regards, The AI2S2 2023 organisers

1 0

Call for Submission: ACM Multimedia 2023 Workshop On Generative AI And Multimodal LLM For Finance (GenAI4Finance)
by puneetm＠umd.edu 09 Aug '23

09 Aug '23

Hi all, Announcing the ACM Multimedia 2023 Workshop On Generative AI And Multimodal LLM For Finance (GenAI4Finance). Looking forward to receiving exciting submissions on the broad theme of algorithms and applications of Generative AI (Diffusion Models, GANS, LLMs, Transformers, ChatGPT, Opensource LLMs, etc) in Finance! Paper deadline: Aug 10 '23 Website: https://generativeai4finance-mm23.github.io/ Papers accepted at the conference will be published in ACM proceedings and presented at ACM MM in Ottawa, Canada. Remote as well as in-person presentations are allowed! Jointly organized by researchers from the University of Maryland Georgia Institute of Technology Adobe MBZUAI (Mohamed bin Zayed University of Artificial Intelligence) and The Philipp University of Marburg. CALL FOR PAPERS: About The GenAI4Finance Workshop Trading and investments in financial markets are useful mechanisms for wealth creation and poverty alleviation. Financial forecasting is an essential task that helps investors make sound investment decisions. It can assist users in better predicting price changes in stocks and currencies along with managing risk to reduce chances of adverse losses and bankruptcies. The world of finance is characterized by millions of people trading numerous assets in complex transactions, giving rise to an overwhelming amount of data that can indicate the future direction of stocks, equities, bonds, currencies, crypto coins, and non-fungible tokens (NFTs). Traditional statistical techniques have been extensively studied for stock market trading in developed economies. However, these techniques hold less relevance with rising automation in trading, the interconnected nature of the economies, and the advent of high-risk, high-reward cryptocurrencies and decentralized financial products like NFTs. Today, the world of finance has been brought ever closer to technology as technological innovations become a core part of financial systems. Hence, there is a need to systematically analyze how advances in Generative Artificial Intelligence and Large Language Modeling can help citizen investors, professional investment companies and novice enthusiasts better understand the financial markets. Relevance Of Multimodal LLMs And Generative AI In Finance The biggest hurdle in processing unstructured data through LLMs like ChatGPT, GPT-3.5, Multimodal GPT-4, PaLM, and others is their opaque nature that they learn from large internet-sized corpora. Lack of understanding of what, when, and how these models can be utilized for financial applications remains an unsolved problem. While these models have shown tremendous success in NLP and document processing tasks, their study on financial knowledge has been very limited. The difficulty in automatically capturing real-time stock market information by these LLMs impedes their successful application for traders and finance practitioners. The learning abilities of LLMs on streaming news sentiment expressed through different multimedia has been barely touched upon by researchers in finance. Artificial Intelligence based methods can help bridge that gap between the two domains through inter-sectional studies. Even though there have been scattered studies in understanding these challenges, there is a lack of a unified forum to focus on the umbrella theme of generative AI + multimodal language modeling for financial forecasting. We expect this workshop to attract researchers from various perspectives to foster research on the intersection of multimodal AI and financial forecasting. We also plan to organize one shared task to encourage practitioners and researchers to gain a deeper understanding of the problems and come up with novel solutions. Topics: This workshop will hold a research track and a shared task. The research track aims to explore recent advances and challenges of Multimodal Generative LLMs for finance. Researchers from artificial intelligence, computer vision, speech processing, natural language processing, data mining, statistics, optimization, and other fields are invited to submit papers on recent advances, resources, tools, and challenges on the broad theme of Multimodal AI for finance. The topics of the workshop include but are not limited to the following: Generative AI applications in finance LLM evaluation for finance Fine-tuning LLMs for finance Pre-training strategies for Financial LLMs Retrieval augmentations for interpretable finance Hallucinations in Financial LLMs and Generative AI Machine Learning for Time Series Data Audio-Video Processing for Financial Unstructured Data Processing text transcript with audio/video for audio-visual-textual alignment, information extraction, salient event detection, entity linking Conversational dialogue modeling using LLMs Natural Language Processing Applications in Finance Named-entity recognition, relationship extraction, ontology learning in financial documents Multimodal financial knowledge discovery Data construction, acquisition, augmentation, feature engineering, and analysis for investment and risk management Bias analysis and mitigation in financial LLMs and datasets Interpretability and explainability for financial Generative AI models and Multimodal LLMs Privacy-preserving AI for finance Video understanding of financial multimedia Vision + language and/or other modalities Time series modeling for financial applications Important Dates - June 1, 2023: ACM MM 2023 Workshop Website Open - August 10, 2023: Submission Deadline - August 15, 2023: Paper notification - August 18, 2023: Camera-Ready Deadline All deadlines are “anywhere on earth” (UTC-12) Submission: Authors are invited to submit their unpublished work that represents novel research. The papers should be written in English using the ACM Multimedia-23 author kit and follow the ACM Multimedia 2023 formatting guidelines. Authors can also submit supplementary materials, including technical appendices, source codes, datasets, and multimedia appendices. All submissions, including the main paper and its supplementary materials, should be fully anonymized. For more information on formatting and anonymity guidelines, please refer to ACM Multimedia 2023 call for papers page. Reviewing: At least two reviewers with matching technical expertise will review each paper. Each paper should be accompanied by one workshop registration. Multiple submissions of the same paper to more ACM Multimedia-2023 workshops are forbidden. All papers will be double-blind peer-reviewed. The workshop accepts both long papers and short papers: Short Paper: Up to 4 pages of content, plus unlimited pages for references and appendix. Upon acceptance, the authors are provided with 1 more page to address the reviewer's comments. Long Paper: Up to 8 pages of content, plus unlimited pages for references and appendix. Upon acceptance, the authors are provided with 1 more page to address the reviewer's comments. Two reviewers with the same technical expertise will review each paper. Authors of the accepted papers will present their work in either the Oral or Poster session. All accepted papers will appear in the workshop proceedings to be published in ACM Multimedia 2023 proceedings. The authors will keep the copyright of their published papers. The paper must be submitted using EasyChair (https://easychair.org/my/conference?conf=genai4finance).

1 0

[JOBS] PhD and permanent Postdoc positions in Argumentation
by Jacky Visser (Staff) 08 Aug '23

08 Aug '23

In addition to advertising a permanent postdoctoral research position (see arg.tech/2023PDRA01<http://arg.tech/2023PDRA01>, deadline 14 August 2023) the Centre for Argument Technology is now also opening a call for applications for fully-funded PhD studentships. Applications are invited across the full range of interdisciplinary research in the Centre, with more details available at arg.tech/2023PHD01<http://arg.tech/2023PHD01>. Closing date for the PhD studentships is 23 September 2023. The University of Dundee is a registered Scottish Charity, No: SC015096

1 0

math and language/text data (continued from thread "Re: Re: Any literature about tensors-based corpora NLP research with actual examples (and homework ;-)) you would suggest? ...")
by Ada Wan 08 Aug '23

08 Aug '23

[resending since it looked like my reply bounced due to size of thread] ---------- Forwarded message --------- From: Ada Wan <adawan919(a)gmail.com> Date: Mon, Aug 7, 2023 at 3:46 PM Subject: Re: [Corpora-List] Re: Any literature about tensors-based corpora NLP research with actual examples (and homework ;-)) you would suggest? ... To: Anil Singh <anil.phdcl(a)gmail.com> Cc: Hesham Haroon <heshamharoon19(a)gmail.com>, corpora < corpora(a)list.elra.info> Hi Anil Just a couple of quick points for now: 1. re encoding etc.: I know how involved and messy this "encoding" and "clean-up" issues are/were and could be for the non-ASCII world. I wish people who work on multilngual NLP and/or language technologies would give these issues more attention. [Iiuc, these intros cover the standardized bit from Unicode (also good for other viewers at home, I mean, readers of this thread :) ): https://www.unicode.org/notes/tn10/indic-overview.pdf https://www.unicode.org/faq/indic.html] Re "there is still no universally accepted way to actually use these input methods" and "Most Indians, for example, on social media and even in emails type in some pseudo-phonetic way using Roman letters with the QWERTY keyboard": actually this is true in many other parts of the non-ASCII world. Even in DE and FR, one has "gotten used to" circumventing the ASCII-conventions with alternate spellings, e.g. spelling out umlauts, dropping the accents.... Between i) getting people to just use script the "standardized way" and ii) weaning people from wordhood --- I wonder which would be harder. :) But I regard i) as a vivid showcase of human creativity, I don't think there is anything to "correct". As with ii), it could be interpreted as baggage accompanying "ASCII-centrism". Language is (re-)productive in nature anyway, there will always be novel elements or ways of expression that could be ahead of standardization. I don't think we should dictate how people write or express themselves digitally (by asking them to adhere to standards or grammar), but, instead, we (as technologists) should use good methods and technologies to deal with the data that we have. (N.B. one doesn't write in "words". One just writes. "Word count" is arbitrary, but character count is not --- unless one is in charge of the lower-level/technical design of such.) For text-based analyses with "non-standard varieties or input idiosyncrasies", I suppose one would just have to have good methods in order to extract information from these (if so wished). (Btw, iiuc, you are also of the opinion that there isn't much/anything else to work on as far as character encoding for scripts usually used in India or standardization on this front is concerned, correct? If not, please let it be known here. I bet/hope some Unicode folks might be on this thread?) 2. Re word and morphology: I suppose what I am advocating is that not only is there really no word (whatever that means etc.), there is also no morphology --- not as something intrinsic in language nor as a science. I understand that this can be hard to accept for those of us who have had a background in Linguistics or many language-related areas, because morphology has been part of an academic discipline for a while. But the point is that morphological analyses are really just some combination of post-hoc analyses based on traditional philological practices and preferences (i.e. ones with selection bias). It is not a universal decomposition method. 3. Re "but ... Sometimes it is wise to be silent": I'm afraid I'd have to disagree with you here. Before my experiences these past years, I had never wanted to be "famous"/"infamous". I had just wanted to work on my research. But in the course of these few difficult years, I realized part of the "complexity" lied in something that might have had made my whole life and my "(un-)career"/"atypical career path" difficult --- our language interpretation and our community's/communities' attitudes towards language (again, whatever "language" means!). 4. Re grammar: much of my "'no grammar' campaign" comes from my observation that many people seem to have lost it with language. The "word"/"grammar"-hacking phenomena, esp. in Linguistics/CL/NLP, exacerbate, otoh, our dependency on grammar, otoh, that on words --- when neither of these is real or necessary. And then the more I see these people advertising their work on social media, the more likely it is for them to misinform and influence others (may these be the general public or computer scientists who do not have much of a background in language or in the science of language), warping the their relationship with language. (The results of my resolving "morphological complexity" in "Fairness in Representation" could have been hard to experience for some.) Re "I don't say it is necessary, but I see it as one possible model to describe language": Ok, to this "grammar is useful" argument: esp. considering how grammar and linguistics (as a subject) both suffered from selection bias, and both have some intrinsic problems in their approach with "judgments", even if such models might work sometimes in describing some data, I don't think it'd be ethical to continue pursuing/developing grammatical methods. When dealing with texts, character n-grams (based on actual data!!)** and their probabilities should suffice as a good description --- insofar as one actually provides a description. There is no need to call it "grammar". It just invokes tons of unnecessary dependencies. **And when over time and across datasets, one is able to generalize truths (may these be for language/communication/information/computation/mathematics/...) from these statistical accounts that hold, then there is a potential for good theories. But as of now, even in Linguistics, there is still a tendency to cherry-pick, in e.g. discarding/disregarding whitespaces. And esp. considering how little data (as in, so few good/verified datasets) we have in general, there is still quite some work to do for all when it comes to good data scientific theories. Re "which can be useful for some -- like educational -- purposes if used in the right way": how grammar has been too heavily used in education is more of an ailment than a remedy or healthy direction. I think the only way for grammar to survive is to regard it as "style guides". Re "particularly with non-native speakers of English ... sometimes your patience is severely tested": to that: yes, I understand/empathize, but no --- in that we ought to change how writing/language is perceived. Depending on the venue/publication, I think sometimes we ought to relax a bit with other's stylistic performance. The "non-native speakers of X" has been a plague in Linguistics for a while now. We have almost paved our way to some next-gen eugenics with that. Most of us on this mailing list are not submitting to non-scientific literary publications, I'd prefer better science and content to better writing styles at any time. Re "magic": I think sometimes I do get some "magical" results with higher-dimensional models. So there is some "magic" (or ((math/truths)+data)? :) ) that is not so obvious. That, I am always glad to look further into. Best Ada On Sat, Aug 5, 2023 at 6:26 PM Anil Singh <anil.phdcl(a)gmail.com> wrote: > On Sat, Aug 5, 2023 at 6:56 PM Ada Wan <adawan919(a)gmail.com> wrote: > >> Hi Anil >> >> Thanks for your comments. (And thanks for reading my work.) >> >> Yeah, there is a lot that one has to pay attention to when it comes to >> what "textual computing" entails (and to which extent it "exists"). Beyond >> "grammar" definitely. But experienced CL folks should know that. (Is this >> you btw: https://scholar.google.com/citations?user=QKnpUbgAAAAJ? >> > > Yes, that's me, for the better or for the worse. > > >> If not, do you have a webpage for your work? Nice to e-meet you either >> way!) >> >> > Thank you. > > Re "I know first hand the problems in doing NLP for low resource languages >> which are related to text encodings": >> which specific languages/varieties are you referring to here? If the >> issue lies in the script not having been encoded, one can contact SEI about >> it (https://linguistics.berkeley.edu/sei/)? I'm always interested in >> knowing what hasn't been encoded. Are the scripts on this list ( >> https://linguistics.berkeley.edu/sei/scripts-not-encoded.html)? >> >> > Well, that's a long story. It is related to the history of adaptation of > computers by the public at large in India. The really difficult part is not > about scripts being encoded. Instead, it is about a script being > over-encoded or encoded in a non-standard way. And the lack of adoption of > standard encodings and input methods. Just to give one example, even though > a single encoding (called ISCII) for all Brahmi-origin scripts of India was > created officially, most people were unaware of it or didn't use it for so > many reasons. One major reason being that it was not supported on Operating > Systems, including Windows (which was anyway developed many years after > creation of ISCII). Input methods and rendering engines for it were not > available. You had to have a special terminal to use it, but that was text > only terminal, used mainly in research centers and perhaps for some very > limited official purposes. And computers, as far as the general public was > concerned, were most commonly used for DeskTop Publishing (which became > part of Indian languages as "DTP"). These non-standard encodings were > mainly font-encodings, just to enable proper rendering of text for > publishing. One of the most popular 'encodings' was based on the Remington > typewriter for Hindi. Another was based on mostly phonetic mapping from > Roman to Devanagari. Other languages which did not use Devanagari also had > their own non-standard encodings, often multiple encodings. The reason > these became popular was that they enabled people to type in Indian > languages and see the text rendered properly, since no other option was > available and they understandably didn't really care about it being > standard or not. It wasn't until recently that Indic scripts were properly > supported by any OS's. It is possible that even now, when Unicode is > supported on most OS's and input methods are available as part of OS's, > there are people still using non-standard encodings. Even now, you can come > across problems related to either input methods or rendering for Indic > scripts on OS's. And most importantly, there is still no universally > accepted way to actually use these input methods. Most Indians, for > example, on social media and even in emails type in some pseudo-phonetic > way using Roman letters with the QWERTY keyboard. Typing in Indian > languages using Indic scripts is still a specialized skill. > > The result of all this is that when you try to collect data for low > resource languages, including major languages of India, there may be a lot > of data -- or perhaps even all the data, depending on the language -- which > is in some non-standard ad-hoc encoding which has non-trivial mapping with > Unicode. This is difficult partly because non-standard encodings are often > based on glyphs, rather than actual units of the script. So, to be able to > use it you need a perfect encoding converter to get the text in Unicode > (UTF-8). Such converters have been there for a long time, but since they > were difficult to create, they were/are proprietary and not available even > to researchers in most cases. It seems a pretty good OCR system has been > developed for Indic scripts/languages, but I have not yet had the chance to > try it. > > For example, I am currently (for the last few years) working on Bhojpuri, > Magahi and Maithili. When we tried to collect data for these languages, > there was the same problem, which is actually not really a problem for the > general public because their purpose is served by these non-standard > encodings, but for NLP/CL you face difficulty in getting the data in a > usable form. > > This is just a brief overview and I also don't really know the full extent > of it, in the sense that I don't have a comprehensive list of such > non-standard encodings for all Indic scripts. > > >> Re the unpublished paper (on a computational typology of writing >> systems?): >> when and to where (as in, which venues/publications) did you submit it? >> I remember one of my first term papers from the 90s being on the >> phonological system of written Cantonese (or sth like that --- don't >> remember my wild days), the prof told me it wasn't "exactly linguistics"... >> >> I had submitted to the journal Written Language and Literacy in 2009. It > was actually mostly my mistake that I didn't submit a revised version of > the paper as I was going through a difficult period then. > > >> Re "on building an encoding converter that will work for all 'encodings' >> used for Indian languages": >> this sounds interesting! >> >> > Yes, I still sometimes wish I could build it. > > >> Re "I too wish there was a good comprehensive history text encodings, >> including non-standard ad-hoc encodings": >> what do you mean by that --- history of text encodings or historical text >> encodings? >> After my discoveries from recent years, when my "mental model" towards >> what's been practiced in the language space (esp. in CL/NLP) finally >> *completely *shifted, I had wanted to host (or co-host) a tutorial on >> character encoding for those who might be under-informed on the matter >> (including but not limited to the "grammaroholics" (esp. the CL/NLP >> practitioners who seem to be stuck doing grammar, even in the context of >> computing) --- there are so many of them! :) ) >> >> > I mostly meant the non-standard 'encodings' (really just ad-hoc mappings) > to serve someone's current purpose. To fully understand the situation, you > have to be familiar with social-political-economic-etc. aspects of the > language situation in India. > > >> Re "word level language identification": >> I don't do "words" anymore. In that 2016 TBLID paper of mine, I >> (regrettably) was still going with the flow in under-reporting on >> tokenization procedures (like what many "cool" ML papers did). But "words" >> do certainly shape the results! I'm really forward to everyone working with >> full-vocabulary, pure character or byte formats (depending on the task), >> while being 100% aware of statistics. Things can be much more transparent >> and easily replicable/reproducible that way anyway. >> >> > Well, I used the word 'word' as just a shorthand for space separated > segments. In my PhD thesis, I had also argued against word being the unit > of computational processing or whatever you call it. I had called the unit > Extra-Lexical Unit, consisting of a core morpheme and inflectional parts. I > realize now that even that may not necessarily work for languages with > highly fusional morphology. But, something like this is now the preferred > unit of morphological processing, as in the CoNLL shared tasks and > UniMorph. I also realize that I could not have been the first to come to > this conclusion. > > >> Re "We have to be tolerant of what you call bad research for various >> unavoidable reasons. Research is not what it used to be": >> No, I think one should just call out bad research and stop doing it. I >> wouldn't want students to burn their midnight oil working hard for nothing. >> Bad research warps also expectations and standards, in other sectors as >> well (education, healthcare, commerce... etc.). Science, as in the pursuit >> of truth and clarity, is and should be the number 1 priority of any decent >> research. (In my opinion, market research or research for marketing >> purposes should be all consolidated into one track/venue if they lack >> scientific quality.) I agree research is not what it used to be --- but in >> the sense that the quality is much worse in general, much hacking around >> with minor, incremental improvements. Like in the case of "textual >> computing", people are "grammar"-hacking. >> >> > I completely agree with you, but ... Sometimes it is wise to be silent. > > >> Re *better ... gender representation": >> hhmm... I'm not so sure about that. >> >> > You are a better judge of that. I just shared my opinion, which may not be > completely free from bias, although I do try. > > >> Re "About grammar, I have come to think of it as a kind of language model >> for describing some linguistic phenomenon": >> nah, grammar not necessary. >> >> > I don't say it is necessary, but I see it as one possible model to > describe language, which can be useful for some -- like educational -- > purposes if used in the right way. > > >> Re grammaroholic reviewers: >> yeah, there are tons of those in the CL/NLP space. I think many of them >> are only willing and/or able to critique on grammar. Explicit is that it >> shows that they don't want to check one's math and code --- besides, when >> most work on "words" anyway, there is a limit to how things are >> replicable/reproducible, esp. if on a different dataset. The implicit bit, >> however, is that I think there is some latent intent to introduce/reinforce >> the influence of "grammar" into the computing space. That, I do not agree >> with at all. >> >> > I should confess that I sometimes am guilty of that (pointing out > grammatical mistakes) myself. However, the situation is complicated in > countries like India due to historical and other reasons. I think the > papers should at least be in a condition that they can be understood > roughly as intended. This may not always be the case, particularly with > non-native speakers of English, or people who are not yet speakers/writers > of English at all. Now, perhaps no one knows better than myself that it is > not really their fault completely, but as a reviewer, sometimes your > patience is severely tested. > > >> Re "magic": >> yes, once one gets over the hype, it's just work. >> >> > True, but what I said is based on where I am coming from (as in, to this > position), which will take a really long time to explain. Of course, I > don't literally mean magic. > > Re "I have no experience of field work at all and that I regret, but it is >> partly because I am not a social creature": >> one can be doing implicit and unofficial "fieldwork" everyday if one pays >> attention to how language is used. >> >> > That indeed I do all the time. I meant official fieldwork. >

1 1

Call For Participation: Machine Translation for Indian Languages (MTIL)
by Surupendu Gangopadhyay 08 Aug '23

08 Aug '23

Apologies for multiple posting *********************************** *------------------------------------------------------------------------------------------------Machine Translation for Indian Languages (MTIL) 2023------------------------------------------------------------------------------------------------* We invite all IR and NLP researchers and enthusiasts to participate in the MTIL track (https://mtilfire.github.io/mtil/2023/) held in conjunction with the Forum for Information Retrieval Evaluation (FIRE) 2023 ( http://fire.irsi.res.in/). Indian languages have many linguistic complexities. Though some Indian languages share syntactic similarities, some possess intricate morphological structures. At the same time, some Indian languages are low-resource. Therefore the machine translation models should address these unique challenges in translating between Indian languages. The MTIL track consists of two tasks: 1. *General Translation Task (Task 1):* Task participants should build a machine translation model to translate sentences of the following language pairs: 1. Hindi-Gujarati 2. Hindi-Kannada 3. Kannada-Hindi 4. Hindi-Odia 5. Odia-Hindi 6. Hindi-Punjabi 7. Punjabi-Hindi 8. Hindi-Sindhi 9. Urdu-Kashmiri 10. Telugu-Hindi 11. Hindi-Telugu 12. Urdu-Hindi 13. Hindi-Urdu 2. *Domain Specific Translation Task (Task 2)*: Task participants will build machine translation models for Governance and Healthcare domains. 1. Healthcare: a. Hindi-Gujarati b. Kannada-Hindi c. Hindi-Odia d. Odia-Hindi e. Hindi-Punjabi f. Kannada-Hindi 2. Governance: a. Hindi-Gujarati b. Kannada-Hindi c. Hindi-Odia d. Odia-Hindi e. Hindi-Punjabi f. Kannada-Hindi *Dataset:* The primary source of parallel language pairs is Bharat Parallel Corpus Collection (BPCC), released by AI4Bharat (https://ai4bharat.iitm.ac.in/bpcc ). Participants are encouraged to add datasets of their choice, including parallel corpora and monolingual datasets, to train their models. More information on registration and participation in the track can be found here: https://mtilfire.github.io/mtil/2023/ This track is being done in association with BHASHINI ( https://bhashini.gov.in/) *Organisers* - Prasenjit Majumder, DAIICT Gandhinagar,India and TCG CREST, Kolkata,India - Arafat Ahsan, IIIT-Hyderabad,India - Asif Ekbal, IIT-Patna,India - Saran Pandian, DAIICT Gandhinagar,India - Ramakrishna Appicharla, IIT-Patna ,India - Surupendu Gangopadhyay, DAIICT Gandhinagar,India - Ganesh Epili, DAIICT Gandhinagar,India - Dreamy Pujara, DAIICT Gandhinagar,India - Misha Patel, DAIICT Gandhinagar,India - Aayushi Patel, DAIICT Gandhinagar,India - Bhargav Dave, DAIICT Gandhinagar,India - Mukesh Jha, DAIICT Gandhinagar,India

1 0

2026

2025

2024

2023

2022

Corpora